[==========] Running 15 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 13 tests from LeaderRebalancerTest
[ RUN ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260807 01:45:13.029935 24592 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.24.4.62:36655
I20260807 01:45:13.033638 24592 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 01:45:13.035573 24592 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 01:45:13.061724 24604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 01:45:13.061785 24606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 01:45:13.061724 24603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:45:13.063123 24592 server_base.cc:1061] running on GCE node
I20260807 01:45:14.293915 24592 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 01:45:14.294353 24592 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 01:45:14.294699 24592 hybrid_clock.cc:648] HybridClock initialized: now 1786067114294668 us; error 0 us; skew 500 ppm
I20260807 01:45:14.296386 24592 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 01:45:14.303972 24592 webserver.cc:533] Webserver started at http://127.24.4.62:39445/ using document root <none> and password file <none>
I20260807 01:45:14.305727 24592 fs_manager.cc:362] Metadata directory not provided
I20260807 01:45:14.306000 24592 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 01:45:14.306802 24592 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 01:45:14.314127 24592 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786067112798834-24592-0/minicluster-data/master-0-root/instance:
uuid: "ec32210d703d4425858186c0b0c1d32e"
format_stamp: "Formatted at 2026-08-07 01:45:14 on dist-test-slave-zj8q"
I20260807 01:45:14.327188 24592 fs_manager.cc:696] Time spent creating directory manager: real 0.011s user 0.006s sys 0.004s
I20260807 01:45:14.340338 24615 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 01:45:14.342876 24592 fs_manager.cc:730] Time spent opening block manager: real 0.011s user 0.003s sys 0.003s
I20260807 01:45:14.343426 24592 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786067112798834-24592-0/minicluster-data/master-0-root
uuid: "ec32210d703d4425858186c0b0c1d32e"
format_stamp: "Formatted at 2026-08-07 01:45:14 on dist-test-slave-zj8q"
I20260807 01:45:14.343820 24592 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786067112798834-24592-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786067112798834-24592-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786067112798834-24592-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 01:45:14.419850 24592 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 01:45:14.421841 24592 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 01:45:14.422461 24592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 01:45:14.604717 24592 rpc_server.cc:307] RPC server started. Bound to: 127.24.4.62:36655
I20260807 01:45:14.605314 24690 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.4.62:36655 every 8 connection(s)
I20260807 01:45:14.633517 24691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:14.692322 24691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e: Bootstrap starting.
I20260807 01:45:14.702126 24691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:14.706625 24691 log.cc:826] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e: Log is configured to *not* fsync() on all Append() calls
I20260807 01:45:14.720357 24691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e: No bootstrap required, opened a new log
I20260807 01:45:14.776681 24691 raft_consensus.cc:359] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec32210d703d4425858186c0b0c1d32e" member_type: VOTER }
I20260807 01:45:14.778103 24691 raft_consensus.cc:385] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:14.778757 24691 raft_consensus.cc:740] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec32210d703d4425858186c0b0c1d32e, State: Initialized, Role: FOLLOWER
I20260807 01:45:14.781076 24691 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec32210d703d4425858186c0b0c1d32e" member_type: VOTER }
I20260807 01:45:14.782636 24691 raft_consensus.cc:399] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 01:45:14.783205 24691 raft_consensus.cc:493] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 01:45:14.783819 24691 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:14.790333 24691 raft_consensus.cc:515] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec32210d703d4425858186c0b0c1d32e" member_type: VOTER }
I20260807 01:45:14.792281 24691 leader_election.cc:304] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec32210d703d4425858186c0b0c1d32e; no voters:
I20260807 01:45:14.798128 24691 leader_election.cc:290] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 01:45:14.799476 24694 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:45:14.816301 24694 raft_consensus.cc:697] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e [term 1 LEADER]: Becoming Leader. State: Replica: ec32210d703d4425858186c0b0c1d32e, State: Running, Role: LEADER
I20260807 01:45:14.818872 24694 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec32210d703d4425858186c0b0c1d32e" member_type: VOTER }
I20260807 01:45:14.820740 24691 sys_catalog.cc:565] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 01:45:14.854804 24697 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec32210d703d4425858186c0b0c1d32e. Latest consensus state: current_term: 1 leader_uuid: "ec32210d703d4425858186c0b0c1d32e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec32210d703d4425858186c0b0c1d32e" member_type: VOTER } }
I20260807 01:45:14.856361 24697 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e [sys.catalog]: This master's current role is: LEADER
I20260807 01:45:14.859920 24695 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec32210d703d4425858186c0b0c1d32e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec32210d703d4425858186c0b0c1d32e" member_type: VOTER } }
I20260807 01:45:14.861645 24695 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e [sys.catalog]: This master's current role is: LEADER
I20260807 01:45:14.896493 24707 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260807 01:45:14.930097 24707 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260807 01:45:14.956411 24592 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 01:45:14.984239 24707 catalog_manager.cc:1383] Generated new cluster ID: 0f846de4a85e4f9eb55613e940a90dba
I20260807 01:45:14.984840 24707 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260807 01:45:15.012971 24707 catalog_manager.cc:1406] Generated new certificate authority record
I20260807 01:45:15.015326 24707 catalog_manager.cc:1540] Loading token signing keys...
I20260807 01:45:15.039062 24707 catalog_manager.cc:6070] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e: Generated new TSK 0
I20260807 01:45:15.040889 24707 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260807 01:45:15.108837 24592 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 01:45:15.129618 24721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 01:45:15.136817 24723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:45:15.146080 24592 server_base.cc:1061] running on GCE node
W20260807 01:45:15.146607 24727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:45:15.147655 24592 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 01:45:15.147892 24592 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 01:45:15.148074 24592 hybrid_clock.cc:648] HybridClock initialized: now 1786067115148055 us; error 0 us; skew 500 ppm
I20260807 01:45:15.148711 24592 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 01:45:15.154882 24592 webserver.cc:533] Webserver started at http://127.24.4.1:43587/ using document root <none> and password file <none>
I20260807 01:45:15.155651 24592 fs_manager.cc:362] Metadata directory not provided
I20260807 01:45:15.155941 24592 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 01:45:15.156250 24592 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 01:45:15.157685 24592 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786067112798834-24592-0/minicluster-data/ts-0-root/instance:
uuid: "9cb56177a511438f9f4a8c9ad5d64852"
format_stamp: "Formatted at 2026-08-07 01:45:15 on dist-test-slave-zj8q"
I20260807 01:45:15.167114 24592 fs_manager.cc:696] Time spent creating directory manager: real 0.009s user 0.000s sys 0.008s
I20260807 01:45:15.174590 24737 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 01:45:15.175798 24592 fs_manager.cc:730] Time spent opening block manager: real 0.006s user 0.004s sys 0.003s
I20260807 01:45:15.176261 24592 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786067112798834-24592-0/minicluster-data/ts-0-root
uuid: "9cb56177a511438f9f4a8c9ad5d64852"
format_stamp: "Formatted at 2026-08-07 01:45:15 on dist-test-slave-zj8q"
I20260807 01:45:15.176714 24592 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786067112798834-24592-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786067112798834-24592-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786067112798834-24592-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 01:45:15.217984 24592 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 01:45:15.221905 24592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 01:45:15.230508 24592 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 01:45:15.243306 24592 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 01:45:15.243750 24592 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260807 01:45:15.244163 24592 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 01:45:15.244428 24592 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 01:45:15.502378 24592 rpc_server.cc:307] RPC server started. Bound to: 127.24.4.1:43099
I20260807 01:45:15.502656 24835 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.4.1:43099 every 8 connection(s)
I20260807 01:45:15.518154 24592 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 01:45:15.545046 24848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:45:15.558249 24592 server_base.cc:1061] running on GCE node
W20260807 01:45:15.559757 24852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 01:45:15.564242 24849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:45:15.565796 24592 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 01:45:15.566190 24592 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 01:45:15.566450 24592 hybrid_clock.cc:648] HybridClock initialized: now 1786067115566428 us; error 0 us; skew 500 ppm
I20260807 01:45:15.567317 24592 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 01:45:15.570137 24839 heartbeater.cc:344] Connected to a master server at 127.24.4.62:36655
I20260807 01:45:15.570892 24839 heartbeater.cc:461] Registering TS with master...
I20260807 01:45:15.570926 24592 webserver.cc:533] Webserver started at http://127.24.4.2:40885/ using document root <none> and password file <none>
I20260807 01:45:15.571709 24592 fs_manager.cc:362] Metadata directory not provided
I20260807 01:45:15.571982 24592 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 01:45:15.572332 24592 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 01:45:15.572347 24839 heartbeater.cc:507] Master 127.24.4.62:36655 requested a full tablet report, sending...
I20260807 01:45:15.574009 24592 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786067112798834-24592-0/minicluster-data/ts-1-root/instance:
uuid: "f22c8ad8eba94330950285be7e2a09a8"
format_stamp: "Formatted at 2026-08-07 01:45:15 on dist-test-slave-zj8q"
I20260807 01:45:15.577944 24639 ts_manager.cc:194] Registered new tserver with Master: 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099)
I20260807 01:45:15.582145 24639 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54884
I20260807 01:45:15.583357 24592 fs_manager.cc:696] Time spent creating directory manager: real 0.009s user 0.005s sys 0.001s
I20260807 01:45:15.589643 24859 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 01:45:15.590922 24592 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 01:45:15.591343 24592 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786067112798834-24592-0/minicluster-data/ts-1-root
uuid: "f22c8ad8eba94330950285be7e2a09a8"
format_stamp: "Formatted at 2026-08-07 01:45:15 on dist-test-slave-zj8q"
I20260807 01:45:15.591753 24592 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786067112798834-24592-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786067112798834-24592-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786067112798834-24592-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 01:45:15.622786 24592 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 01:45:15.624790 24592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 01:45:15.628399 24592 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 01:45:15.633283 24592 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 01:45:15.633522 24592 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 01:45:15.633811 24592 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 01:45:15.633980 24592 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 01:45:16.016225 24592 rpc_server.cc:307] RPC server started. Bound to: 127.24.4.2:44449
I20260807 01:45:16.016948 24949 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.4.2:44449 every 8 connection(s)
I20260807 01:45:16.047921 24592 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 01:45:16.066811 24955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:45:16.067834 24950 heartbeater.cc:344] Connected to a master server at 127.24.4.62:36655
I20260807 01:45:16.068318 24950 heartbeater.cc:461] Registering TS with master...
I20260807 01:45:16.069494 24950 heartbeater.cc:507] Master 127.24.4.62:36655 requested a full tablet report, sending...
W20260807 01:45:16.073071 24956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:45:16.073755 24639 ts_manager.cc:194] Registered new tserver with Master: f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449)
I20260807 01:45:16.077445 24639 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54898
W20260807 01:45:16.088892 24959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:45:16.095279 24592 server_base.cc:1061] running on GCE node
I20260807 01:45:16.097899 24592 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 01:45:16.098256 24592 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 01:45:16.098467 24592 hybrid_clock.cc:648] HybridClock initialized: now 1786067116098443 us; error 0 us; skew 500 ppm
I20260807 01:45:16.099215 24592 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 01:45:16.103488 24592 webserver.cc:533] Webserver started at http://127.24.4.3:46567/ using document root <none> and password file <none>
I20260807 01:45:16.104130 24592 fs_manager.cc:362] Metadata directory not provided
I20260807 01:45:16.104359 24592 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 01:45:16.104653 24592 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 01:45:16.106345 24592 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786067112798834-24592-0/minicluster-data/ts-2-root/instance:
uuid: "07a5248f4bf84ded832b7e261c3768b1"
format_stamp: "Formatted at 2026-08-07 01:45:16 on dist-test-slave-zj8q"
I20260807 01:45:16.112864 24592 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260807 01:45:16.117874 24965 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 01:45:16.118829 24592 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 01:45:16.119211 24592 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786067112798834-24592-0/minicluster-data/ts-2-root
uuid: "07a5248f4bf84ded832b7e261c3768b1"
format_stamp: "Formatted at 2026-08-07 01:45:16 on dist-test-slave-zj8q"
I20260807 01:45:16.119594 24592 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786067112798834-24592-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786067112798834-24592-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786067112798834-24592-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 01:45:16.152315 24592 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 01:45:16.153782 24592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 01:45:16.156603 24592 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 01:45:16.163280 24592 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 01:45:16.163585 24592 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 01:45:16.163885 24592 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 01:45:16.164042 24592 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 01:45:16.587577 24839 heartbeater.cc:499] Master 127.24.4.62:36655 was elected leader, sending a full tablet report...
I20260807 01:45:16.623142 24592 rpc_server.cc:307] RPC server started. Bound to: 127.24.4.3:38161
I20260807 01:45:16.624409 25060 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.4.3:38161 every 8 connection(s)
I20260807 01:45:16.679492 25063 heartbeater.cc:344] Connected to a master server at 127.24.4.62:36655
I20260807 01:45:16.680243 25063 heartbeater.cc:461] Registering TS with master...
I20260807 01:45:16.682757 25063 heartbeater.cc:507] Master 127.24.4.62:36655 requested a full tablet report, sending...
I20260807 01:45:16.688206 24639 ts_manager.cc:194] Registered new tserver with Master: 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:16.688901 24592 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.041211545s
I20260807 01:45:16.692124 24639 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54904
I20260807 01:45:16.863917 24639 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54920:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 01:45:16.876022 24639 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 01:45:17.083992 24950 heartbeater.cc:499] Master 127.24.4.62:36655 was elected leader, sending a full tablet report...
I20260807 01:45:17.246968 24896 tablet_service.cc:1511] Processing CreateTablet for tablet 2791c7f6f9c647df909f735bd72c5a63 (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260807 01:45:17.251976 24895 tablet_service.cc:1511] Processing CreateTablet for tablet 9052c9d021c5487da14be02e7852fb2a (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260807 01:45:17.253165 24898 tablet_service.cc:1511] Processing CreateTablet for tablet f0fd88947c65482b8e9d6b23e96a4db2 (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260807 01:45:17.272025 24777 tablet_service.cc:1511] Processing CreateTablet for tablet 9052c9d021c5487da14be02e7852fb2a (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260807 01:45:17.283805 24896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2791c7f6f9c647df909f735bd72c5a63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.271579 24779 tablet_service.cc:1511] Processing CreateTablet for tablet f0fd88947c65482b8e9d6b23e96a4db2 (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260807 01:45:17.288578 25010 tablet_service.cc:1511] Processing CreateTablet for tablet 2791c7f6f9c647df909f735bd72c5a63 (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260807 01:45:17.283082 24893 tablet_service.cc:1511] Processing CreateTablet for tablet 719f845bd45941dcaa2421158f4b0ebe (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260807 01:45:17.286396 24776 tablet_service.cc:1511] Processing CreateTablet for tablet 719f845bd45941dcaa2421158f4b0ebe (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260807 01:45:17.300051 25010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2791c7f6f9c647df909f735bd72c5a63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.277661 24778 tablet_service.cc:1511] Processing CreateTablet for tablet 2791c7f6f9c647df909f735bd72c5a63 (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260807 01:45:17.296765 24777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9052c9d021c5487da14be02e7852fb2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.294435 25011 tablet_service.cc:1511] Processing CreateTablet for tablet f0fd88947c65482b8e9d6b23e96a4db2 (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260807 01:45:17.298139 25009 tablet_service.cc:1511] Processing CreateTablet for tablet 9052c9d021c5487da14be02e7852fb2a (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260807 01:45:17.321869 24775 tablet_service.cc:1511] Processing CreateTablet for tablet 088984f6ac834d64b99408685df54428 (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260807 01:45:17.322467 25008 tablet_service.cc:1511] Processing CreateTablet for tablet 719f845bd45941dcaa2421158f4b0ebe (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260807 01:45:17.325387 25011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0fd88947c65482b8e9d6b23e96a4db2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.319151 24779 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0fd88947c65482b8e9d6b23e96a4db2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.317348 24892 tablet_service.cc:1511] Processing CreateTablet for tablet 088984f6ac834d64b99408685df54428 (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260807 01:45:17.330492 24775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 088984f6ac834d64b99408685df54428. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.334299 25009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9052c9d021c5487da14be02e7852fb2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.337164 24776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 719f845bd45941dcaa2421158f4b0ebe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.341013 25008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 719f845bd45941dcaa2421158f4b0ebe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.346015 25007 tablet_service.cc:1511] Processing CreateTablet for tablet 088984f6ac834d64b99408685df54428 (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260807 01:45:17.355805 24895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9052c9d021c5487da14be02e7852fb2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.360569 24892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 088984f6ac834d64b99408685df54428. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.366253 24893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 719f845bd45941dcaa2421158f4b0ebe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.367126 24778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2791c7f6f9c647df909f735bd72c5a63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.379983 25007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 088984f6ac834d64b99408685df54428. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.388827 24898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0fd88947c65482b8e9d6b23e96a4db2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.398839 25006 tablet_service.cc:1511] Processing CreateTablet for tablet b606cc1d46c3462f9759d92957b384f0 (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260807 01:45:17.403420 25006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b606cc1d46c3462f9759d92957b384f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.385555 24774 tablet_service.cc:1511] Processing CreateTablet for tablet b606cc1d46c3462f9759d92957b384f0 (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260807 01:45:17.407644 24774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b606cc1d46c3462f9759d92957b384f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.396114 24890 tablet_service.cc:1511] Processing CreateTablet for tablet b606cc1d46c3462f9759d92957b384f0 (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260807 01:45:17.442978 24772 tablet_service.cc:1511] Processing CreateTablet for tablet d3c14e19345c4777a5f505485d89b588 (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260807 01:45:17.464586 24889 tablet_service.cc:1511] Processing CreateTablet for tablet d3c14e19345c4777a5f505485d89b588 (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260807 01:45:17.465307 24890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b606cc1d46c3462f9759d92957b384f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.477056 24772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3c14e19345c4777a5f505485d89b588. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.478097 24889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3c14e19345c4777a5f505485d89b588. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.583423 25005 tablet_service.cc:1511] Processing CreateTablet for tablet d3c14e19345c4777a5f505485d89b588 (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260807 01:45:17.587241 25005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3c14e19345c4777a5f505485d89b588. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.652695 25096 tablet_bootstrap.cc:492] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8: Bootstrap starting.
I20260807 01:45:17.677534 25095 tablet_bootstrap.cc:492] T 9052c9d021c5487da14be02e7852fb2a P 07a5248f4bf84ded832b7e261c3768b1: Bootstrap starting.
I20260807 01:45:17.688069 25094 tablet_bootstrap.cc:492] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852: Bootstrap starting.
I20260807 01:45:17.700201 25063 heartbeater.cc:499] Master 127.24.4.62:36655 was elected leader, sending a full tablet report...
I20260807 01:45:17.703919 24777 tablet_service.cc:1511] Processing CreateTablet for tablet dc90e28bc0a843b1a8485dafefcd4a05 (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260807 01:45:17.720913 24896 tablet_service.cc:1511] Processing CreateTablet for tablet dc90e28bc0a843b1a8485dafefcd4a05 (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260807 01:45:17.725633 24896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc90e28bc0a843b1a8485dafefcd4a05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.728317 24777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc90e28bc0a843b1a8485dafefcd4a05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.760339 25095 tablet_bootstrap.cc:654] T 9052c9d021c5487da14be02e7852fb2a P 07a5248f4bf84ded832b7e261c3768b1: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:17.760758 25007 tablet_service.cc:1511] Processing CreateTablet for tablet dc90e28bc0a843b1a8485dafefcd4a05 (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260807 01:45:17.764132 25007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc90e28bc0a843b1a8485dafefcd4a05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.805433 25094 tablet_bootstrap.cc:654] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:17.809310 25096 tablet_bootstrap.cc:654] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:17.818799 24778 tablet_service.cc:1511] Processing CreateTablet for tablet 78946d0ad61f46a7a81f1592fa57a8d3 (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260807 01:45:17.820590 24778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78946d0ad61f46a7a81f1592fa57a8d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.825032 24898 tablet_service.cc:1511] Processing CreateTablet for tablet 78946d0ad61f46a7a81f1592fa57a8d3 (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260807 01:45:17.826753 24898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78946d0ad61f46a7a81f1592fa57a8d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.836123 25008 tablet_service.cc:1511] Processing CreateTablet for tablet 78946d0ad61f46a7a81f1592fa57a8d3 (DEFAULT_TABLE table=test-workload [id=800ce2610df5499f8d1373665ccb3c69]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260807 01:45:17.838518 25008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78946d0ad61f46a7a81f1592fa57a8d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:17.866042 25095 tablet_bootstrap.cc:492] T 9052c9d021c5487da14be02e7852fb2a P 07a5248f4bf84ded832b7e261c3768b1: No bootstrap required, opened a new log
I20260807 01:45:17.866880 25095 ts_tablet_manager.cc:1403] T 9052c9d021c5487da14be02e7852fb2a P 07a5248f4bf84ded832b7e261c3768b1: Time spent bootstrapping tablet: real 0.227s user 0.028s sys 0.015s
I20260807 01:45:17.873471 25095 raft_consensus.cc:359] T 9052c9d021c5487da14be02e7852fb2a P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:17.874708 25095 raft_consensus.cc:385] T 9052c9d021c5487da14be02e7852fb2a P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:17.875142 25095 raft_consensus.cc:740] T 9052c9d021c5487da14be02e7852fb2a P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a5248f4bf84ded832b7e261c3768b1, State: Initialized, Role: FOLLOWER
I20260807 01:45:17.876113 25095 consensus_queue.cc:260] T 9052c9d021c5487da14be02e7852fb2a P 07a5248f4bf84ded832b7e261c3768b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:17.884670 25096 tablet_bootstrap.cc:492] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8: No bootstrap required, opened a new log
I20260807 01:45:17.885612 25096 ts_tablet_manager.cc:1403] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8: Time spent bootstrapping tablet: real 0.246s user 0.021s sys 0.011s
I20260807 01:45:17.890959 25094 tablet_bootstrap.cc:492] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852: No bootstrap required, opened a new log
I20260807 01:45:17.891681 25094 ts_tablet_manager.cc:1403] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852: Time spent bootstrapping tablet: real 0.255s user 0.021s sys 0.037s
I20260807 01:45:17.901633 25095 ts_tablet_manager.cc:1434] T 9052c9d021c5487da14be02e7852fb2a P 07a5248f4bf84ded832b7e261c3768b1: Time spent starting tablet: real 0.034s user 0.010s sys 0.013s
I20260807 01:45:17.903728 25094 raft_consensus.cc:359] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:17.903661 25096 raft_consensus.cc:359] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:17.904523 25094 raft_consensus.cc:385] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:17.904649 25096 raft_consensus.cc:385] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:17.904928 25094 raft_consensus.cc:740] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb56177a511438f9f4a8c9ad5d64852, State: Initialized, Role: FOLLOWER
I20260807 01:45:17.906473 25095 tablet_bootstrap.cc:492] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1: Bootstrap starting.
I20260807 01:45:17.906033 25094 consensus_queue.cc:260] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:17.910764 25096 raft_consensus.cc:740] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Initialized, Role: FOLLOWER
I20260807 01:45:17.911829 25096 consensus_queue.cc:260] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
W20260807 01:45:17.916838 24708 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 01:45:17.922005 25094 ts_tablet_manager.cc:1434] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852: Time spent starting tablet: real 0.030s user 0.010s sys 0.011s
I20260807 01:45:17.924093 25094 tablet_bootstrap.cc:492] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852: Bootstrap starting.
I20260807 01:45:17.924247 25096 ts_tablet_manager.cc:1434] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8: Time spent starting tablet: real 0.038s user 0.007s sys 0.007s
I20260807 01:45:17.925835 25096 tablet_bootstrap.cc:492] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8: Bootstrap starting.
W20260807 01:45:17.928512 25064 tablet.cc:2406] T 9052c9d021c5487da14be02e7852fb2a P 07a5248f4bf84ded832b7e261c3768b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 01:45:17.948325 25096 tablet_bootstrap.cc:654] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:17.964771 25095 tablet_bootstrap.cc:654] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:17.966781 25094 tablet_bootstrap.cc:654] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:17.991638 25096 tablet_bootstrap.cc:492] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8: No bootstrap required, opened a new log
I20260807 01:45:17.992421 25096 ts_tablet_manager.cc:1403] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8: Time spent bootstrapping tablet: real 0.067s user 0.029s sys 0.026s
I20260807 01:45:17.993161 25095 tablet_bootstrap.cc:492] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1: No bootstrap required, opened a new log
I20260807 01:45:17.993854 25095 ts_tablet_manager.cc:1403] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1: Time spent bootstrapping tablet: real 0.088s user 0.018s sys 0.022s
I20260807 01:45:18.000828 25094 tablet_bootstrap.cc:492] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852: No bootstrap required, opened a new log
I20260807 01:45:18.000025 25096 raft_consensus.cc:359] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.001533 25096 raft_consensus.cc:385] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:18.001860 25094 ts_tablet_manager.cc:1403] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852: Time spent bootstrapping tablet: real 0.078s user 0.005s sys 0.035s
I20260807 01:45:18.002094 25096 raft_consensus.cc:740] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Initialized, Role: FOLLOWER
I20260807 01:45:18.001799 25095 raft_consensus.cc:359] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.003185 25095 raft_consensus.cc:385] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:18.003703 25095 raft_consensus.cc:740] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a5248f4bf84ded832b7e261c3768b1, State: Initialized, Role: FOLLOWER
I20260807 01:45:18.004115 25096 consensus_queue.cc:260] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.005698 25095 consensus_queue.cc:260] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.010483 25094 raft_consensus.cc:359] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.015793 25094 raft_consensus.cc:385] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:18.016523 25096 ts_tablet_manager.cc:1434] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8: Time spent starting tablet: real 0.024s user 0.006s sys 0.012s
I20260807 01:45:18.016379 25094 raft_consensus.cc:740] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb56177a511438f9f4a8c9ad5d64852, State: Initialized, Role: FOLLOWER
I20260807 01:45:18.018465 25096 tablet_bootstrap.cc:492] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8: Bootstrap starting.
I20260807 01:45:18.018029 25094 consensus_queue.cc:260] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.024072 25095 ts_tablet_manager.cc:1434] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1: Time spent starting tablet: real 0.030s user 0.003s sys 0.013s
I20260807 01:45:18.035367 25095 tablet_bootstrap.cc:492] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1: Bootstrap starting.
I20260807 01:45:18.039722 25096 tablet_bootstrap.cc:654] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:18.022799 25094 ts_tablet_manager.cc:1434] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852: Time spent starting tablet: real 0.020s user 0.001s sys 0.018s
I20260807 01:45:18.046245 25094 tablet_bootstrap.cc:492] T 088984f6ac834d64b99408685df54428 P 9cb56177a511438f9f4a8c9ad5d64852: Bootstrap starting.
I20260807 01:45:18.064950 25095 tablet_bootstrap.cc:654] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:18.081882 25096 tablet_bootstrap.cc:492] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8: No bootstrap required, opened a new log
I20260807 01:45:18.083020 25096 ts_tablet_manager.cc:1403] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8: Time spent bootstrapping tablet: real 0.065s user 0.008s sys 0.054s
I20260807 01:45:18.088176 25094 tablet_bootstrap.cc:654] T 088984f6ac834d64b99408685df54428 P 9cb56177a511438f9f4a8c9ad5d64852: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:18.091257 25096 raft_consensus.cc:359] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.092871 25096 raft_consensus.cc:385] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:18.093727 25096 raft_consensus.cc:740] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Initialized, Role: FOLLOWER
I20260807 01:45:18.095796 25096 consensus_queue.cc:260] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.096562 25106 raft_consensus.cc:493] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:18.097735 25106 raft_consensus.cc:515] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.115962 25095 tablet_bootstrap.cc:492] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1: No bootstrap required, opened a new log
I20260807 01:45:18.115123 25106 leader_election.cc:290] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:18.124351 25095 ts_tablet_manager.cc:1403] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1: Time spent bootstrapping tablet: real 0.089s user 0.015s sys 0.046s
I20260807 01:45:18.102104 25096 ts_tablet_manager.cc:1434] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8: Time spent starting tablet: real 0.018s user 0.009s sys 0.005s
I20260807 01:45:18.141934 25096 tablet_bootstrap.cc:492] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8: Bootstrap starting.
I20260807 01:45:18.146821 25095 raft_consensus.cc:359] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.147840 25095 raft_consensus.cc:385] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:18.155093 25107 raft_consensus.cc:493] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:18.155954 25107 raft_consensus.cc:515] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.148231 25095 raft_consensus.cc:740] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a5248f4bf84ded832b7e261c3768b1, State: Initialized, Role: FOLLOWER
I20260807 01:45:18.176390 25095 consensus_queue.cc:260] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.176296 25107 leader_election.cc:290] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:18.192155 25095 ts_tablet_manager.cc:1434] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1: Time spent starting tablet: real 0.067s user 0.004s sys 0.009s
I20260807 01:45:18.198523 25095 tablet_bootstrap.cc:492] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1: Bootstrap starting.
I20260807 01:45:18.225378 25104 raft_consensus.cc:493] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:18.226718 25104 raft_consensus.cc:515] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.235597 25094 tablet_bootstrap.cc:492] T 088984f6ac834d64b99408685df54428 P 9cb56177a511438f9f4a8c9ad5d64852: No bootstrap required, opened a new log
I20260807 01:45:18.237627 25094 ts_tablet_manager.cc:1403] T 088984f6ac834d64b99408685df54428 P 9cb56177a511438f9f4a8c9ad5d64852: Time spent bootstrapping tablet: real 0.192s user 0.026s sys 0.040s
I20260807 01:45:18.242228 25106 raft_consensus.cc:493] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:18.243484 25106 raft_consensus.cc:515] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.260676 25106 leader_election.cc:290] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:18.262281 25104 leader_election.cc:290] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449)
I20260807 01:45:18.266855 25104 raft_consensus.cc:493] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:18.273943 25096 tablet_bootstrap.cc:654] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:18.267478 25104 raft_consensus.cc:515] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.285846 25094 raft_consensus.cc:359] T 088984f6ac834d64b99408685df54428 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.287581 25107 raft_consensus.cc:493] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:18.291273 25107 raft_consensus.cc:515] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.296201 25107 leader_election.cc:290] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:18.302620 25094 raft_consensus.cc:385] T 088984f6ac834d64b99408685df54428 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:18.289799 25104 leader_election.cc:290] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449), 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099)
I20260807 01:45:18.303807 25094 raft_consensus.cc:740] T 088984f6ac834d64b99408685df54428 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb56177a511438f9f4a8c9ad5d64852, State: Initialized, Role: FOLLOWER
I20260807 01:45:18.306432 25094 consensus_queue.cc:260] T 088984f6ac834d64b99408685df54428 P 9cb56177a511438f9f4a8c9ad5d64852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.312412 25094 ts_tablet_manager.cc:1434] T 088984f6ac834d64b99408685df54428 P 9cb56177a511438f9f4a8c9ad5d64852: Time spent starting tablet: real 0.072s user 0.002s sys 0.016s
I20260807 01:45:18.314641 25094 tablet_bootstrap.cc:492] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852: Bootstrap starting.
I20260807 01:45:18.317005 25095 tablet_bootstrap.cc:654] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:18.384500 25107 raft_consensus.cc:493] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:18.388011 24913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9052c9d021c5487da14be02e7852fb2a" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f22c8ad8eba94330950285be7e2a09a8" is_pre_election: true
I20260807 01:45:18.385990 25107 raft_consensus.cc:515] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.395190 25026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9052c9d021c5487da14be02e7852fb2a" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a5248f4bf84ded832b7e261c3768b1" is_pre_election: true
I20260807 01:45:18.396188 25026 raft_consensus.cc:2468] T 9052c9d021c5487da14be02e7852fb2a P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb56177a511438f9f4a8c9ad5d64852 in term 0.
I20260807 01:45:18.401242 24738 leader_election.cc:304] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1, 9cb56177a511438f9f4a8c9ad5d64852; no voters:
I20260807 01:45:18.401566 24795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "088984f6ac834d64b99408685df54428" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852" is_pre_election: true
I20260807 01:45:18.403342 24795 raft_consensus.cc:2468] T 088984f6ac834d64b99408685df54428 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f22c8ad8eba94330950285be7e2a09a8 in term 0.
I20260807 01:45:18.403522 25106 raft_consensus.cc:2804] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:45:18.399255 24913 raft_consensus.cc:2468] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb56177a511438f9f4a8c9ad5d64852 in term 0.
I20260807 01:45:18.404060 25106 raft_consensus.cc:493] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:45:18.405965 25106 raft_consensus.cc:3060] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:18.400594 25025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0fd88947c65482b8e9d6b23e96a4db2" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a5248f4bf84ded832b7e261c3768b1" is_pre_election: true
I20260807 01:45:18.407678 25025 raft_consensus.cc:2468] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb56177a511438f9f4a8c9ad5d64852 in term 0.
I20260807 01:45:18.405654 24912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0fd88947c65482b8e9d6b23e96a4db2" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f22c8ad8eba94330950285be7e2a09a8" is_pre_election: true
I20260807 01:45:18.397917 24800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2791c7f6f9c647df909f735bd72c5a63" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852" is_pre_election: true
I20260807 01:45:18.418490 24792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9052c9d021c5487da14be02e7852fb2a" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852" is_pre_election: true
I20260807 01:45:18.416985 24738 leader_election.cc:304] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1, 9cb56177a511438f9f4a8c9ad5d64852; no voters:
I20260807 01:45:18.421104 25096 tablet_bootstrap.cc:492] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8: No bootstrap required, opened a new log
I20260807 01:45:18.428303 24864 leader_election.cc:304] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9cb56177a511438f9f4a8c9ad5d64852, f22c8ad8eba94330950285be7e2a09a8; no voters:
I20260807 01:45:18.425249 25107 leader_election.cc:290] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:18.432790 25107 raft_consensus.cc:2804] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:45:18.433717 25107 raft_consensus.cc:493] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:45:18.434253 25107 raft_consensus.cc:3060] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:18.431272 25096 ts_tablet_manager.cc:1403] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8: Time spent bootstrapping tablet: real 0.290s user 0.009s sys 0.034s
I20260807 01:45:18.460816 25106 raft_consensus.cc:515] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.449115 25096 raft_consensus.cc:359] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
W20260807 01:45:18.471839 24738 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:38161, user_credentials={real_user=slave}} blocked reactor thread for 58836.7us
I20260807 01:45:18.479530 25096 raft_consensus.cc:385] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:18.481844 25096 raft_consensus.cc:740] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Initialized, Role: FOLLOWER
I20260807 01:45:18.479319 24792 raft_consensus.cc:2393] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f22c8ad8eba94330950285be7e2a09a8 in current term 1: Already voted for candidate 9cb56177a511438f9f4a8c9ad5d64852 in this term.
I20260807 01:45:18.483744 25025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2791c7f6f9c647df909f735bd72c5a63" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a5248f4bf84ded832b7e261c3768b1" is_pre_election: true
I20260807 01:45:18.484747 25025 raft_consensus.cc:2468] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f22c8ad8eba94330950285be7e2a09a8 in term 0.
I20260807 01:45:18.501478 25106 leader_election.cc:290] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 1 election: Requested vote from peers f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:18.501421 25025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "088984f6ac834d64b99408685df54428" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a5248f4bf84ded832b7e261c3768b1" is_pre_election: true
W20260807 01:45:18.497565 24864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:43099, user_credentials={real_user=slave}} blocked reactor thread for 71126us
I20260807 01:45:18.504320 24913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9052c9d021c5487da14be02e7852fb2a" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
W20260807 01:45:18.504622 24742 leader_election.cc:343] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:18.506259 24913 raft_consensus.cc:3060] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:18.500643 25096 consensus_queue.cc:260] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.500311 25137 raft_consensus.cc:2804] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:45:18.507676 25026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9052c9d021c5487da14be02e7852fb2a" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
I20260807 01:45:18.510426 25026 raft_consensus.cc:3060] T 9052c9d021c5487da14be02e7852fb2a P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:18.508776 25107 raft_consensus.cc:515] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
W20260807 01:45:18.507874 24864 leader_election.cc:343] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:18.511822 25137 raft_consensus.cc:493] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:45:18.514418 25137 raft_consensus.cc:3060] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:18.521596 25025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9052c9d021c5487da14be02e7852fb2a" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a5248f4bf84ded832b7e261c3768b1" is_pre_election: true
I20260807 01:45:18.521688 24792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "088984f6ac834d64b99408685df54428" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
I20260807 01:45:18.519703 25095 tablet_bootstrap.cc:492] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1: No bootstrap required, opened a new log
I20260807 01:45:18.524955 25095 ts_tablet_manager.cc:1403] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1: Time spent bootstrapping tablet: real 0.327s user 0.013s sys 0.041s
I20260807 01:45:18.524740 24792 raft_consensus.cc:3060] T 088984f6ac834d64b99408685df54428 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:18.517781 25107 leader_election.cc:290] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 election: Requested vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:18.528652 24861 leader_election.cc:304] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1, f22c8ad8eba94330950285be7e2a09a8; no voters: 9cb56177a511438f9f4a8c9ad5d64852
I20260807 01:45:18.532366 25094 tablet_bootstrap.cc:654] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852: Neither blocks nor log segments found. Creating new log.
W20260807 01:45:18.533576 24861 leader_election.cc:343] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:18.539904 24912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2791c7f6f9c647df909f735bd72c5a63" candidate_uuid: "07a5248f4bf84ded832b7e261c3768b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f22c8ad8eba94330950285be7e2a09a8" is_pre_election: true
I20260807 01:45:18.541061 25096 ts_tablet_manager.cc:1434] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8: Time spent starting tablet: real 0.103s user 0.010s sys 0.006s
I20260807 01:45:18.542397 24861 leader_election.cc:304] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f22c8ad8eba94330950285be7e2a09a8; no voters: 07a5248f4bf84ded832b7e261c3768b1, 9cb56177a511438f9f4a8c9ad5d64852
I20260807 01:45:18.543069 25096 tablet_bootstrap.cc:492] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8: Bootstrap starting.
I20260807 01:45:18.536546 25107 raft_consensus.cc:2804] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:45:18.545393 25107 raft_consensus.cc:493] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:45:18.550715 24911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0fd88947c65482b8e9d6b23e96a4db2" candidate_uuid: "07a5248f4bf84ded832b7e261c3768b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f22c8ad8eba94330950285be7e2a09a8" is_pre_election: true
I20260807 01:45:18.557354 25025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "088984f6ac834d64b99408685df54428" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
W20260807 01:45:18.559988 24861 leader_election.cc:343] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:18.553253 25107 raft_consensus.cc:3060] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Advancing to term 1
W20260807 01:45:18.565953 24974 leader_election.cc:343] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:18.570963 25095 raft_consensus.cc:359] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.572285 25095 raft_consensus.cc:385] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:18.572814 25095 raft_consensus.cc:740] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a5248f4bf84ded832b7e261c3768b1, State: Initialized, Role: FOLLOWER
I20260807 01:45:18.574606 25095 consensus_queue.cc:260] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.575908 25026 raft_consensus.cc:2468] T 9052c9d021c5487da14be02e7852fb2a P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb56177a511438f9f4a8c9ad5d64852 in term 1.
I20260807 01:45:18.578312 24738 leader_election.cc:304] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1, 9cb56177a511438f9f4a8c9ad5d64852; no voters:
I20260807 01:45:18.579836 25106 raft_consensus.cc:2804] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:45:18.573845 25137 raft_consensus.cc:515] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.585343 25106 raft_consensus.cc:697] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 LEADER]: Becoming Leader. State: Replica: 9cb56177a511438f9f4a8c9ad5d64852, State: Running, Role: LEADER
I20260807 01:45:18.590234 25106 consensus_queue.cc:237] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.591054 25107 raft_consensus.cc:515] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.593469 24912 raft_consensus.cc:2393] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 07a5248f4bf84ded832b7e261c3768b1 in current term 1: Already voted for candidate f22c8ad8eba94330950285be7e2a09a8 in this term.
I20260807 01:45:18.598511 24792 raft_consensus.cc:2468] T 088984f6ac834d64b99408685df54428 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f22c8ad8eba94330950285be7e2a09a8 in term 1.
I20260807 01:45:18.599738 25107 leader_election.cc:290] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 election: Requested vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:18.599859 24913 raft_consensus.cc:2468] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb56177a511438f9f4a8c9ad5d64852 in term 1.
I20260807 01:45:18.601374 25142 raft_consensus.cc:2749] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:45:18.605966 25095 ts_tablet_manager.cc:1434] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1: Time spent starting tablet: real 0.080s user 0.006s sys 0.010s
I20260807 01:45:18.607369 25095 tablet_bootstrap.cc:492] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1: Bootstrap starting.
I20260807 01:45:18.608989 25137 leader_election.cc:290] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 1 election: Requested vote from peers f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:18.611151 25025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0fd88947c65482b8e9d6b23e96a4db2" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
I20260807 01:45:18.612007 25025 raft_consensus.cc:3060] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:18.623662 25026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2791c7f6f9c647df909f735bd72c5a63" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
I20260807 01:45:18.624648 25026 raft_consensus.cc:3060] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:18.635931 24792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2791c7f6f9c647df909f735bd72c5a63" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
I20260807 01:45:18.643245 24795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0fd88947c65482b8e9d6b23e96a4db2" candidate_uuid: "07a5248f4bf84ded832b7e261c3768b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852" is_pre_election: true
I20260807 01:45:18.627632 24864 leader_election.cc:304] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9cb56177a511438f9f4a8c9ad5d64852, f22c8ad8eba94330950285be7e2a09a8; no voters: 07a5248f4bf84ded832b7e261c3768b1
I20260807 01:45:18.644541 24795 raft_consensus.cc:2393] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 07a5248f4bf84ded832b7e261c3768b1 in current term 1: Already voted for candidate 9cb56177a511438f9f4a8c9ad5d64852 in this term.
I20260807 01:45:18.648483 25094 tablet_bootstrap.cc:492] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852: No bootstrap required, opened a new log
I20260807 01:45:18.646816 24800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2791c7f6f9c647df909f735bd72c5a63" candidate_uuid: "07a5248f4bf84ded832b7e261c3768b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852" is_pre_election: true
I20260807 01:45:18.649502 25025 raft_consensus.cc:2468] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb56177a511438f9f4a8c9ad5d64852 in term 1.
I20260807 01:45:18.650528 25142 raft_consensus.cc:2804] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:45:18.650600 24973 leader_election.cc:304] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1; no voters: 9cb56177a511438f9f4a8c9ad5d64852, f22c8ad8eba94330950285be7e2a09a8
I20260807 01:45:18.651108 25142 raft_consensus.cc:697] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [term 1 LEADER]: Becoming Leader. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Running, Role: LEADER
I20260807 01:45:18.649155 25094 ts_tablet_manager.cc:1403] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852: Time spent bootstrapping tablet: real 0.335s user 0.016s sys 0.010s
I20260807 01:45:18.653407 25130 raft_consensus.cc:2749] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:45:18.653221 25142 consensus_queue.cc:237] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
W20260807 01:45:18.657685 24973 leader_election.cc:343] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:18.657809 24738 leader_election.cc:304] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1, 9cb56177a511438f9f4a8c9ad5d64852; no voters:
I20260807 01:45:18.658650 24973 leader_election.cc:304] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1; no voters: 9cb56177a511438f9f4a8c9ad5d64852, f22c8ad8eba94330950285be7e2a09a8
I20260807 01:45:18.659420 25106 raft_consensus.cc:2804] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:45:18.660893 25095 tablet_bootstrap.cc:654] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:18.663305 25106 raft_consensus.cc:697] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 LEADER]: Becoming Leader. State: Replica: 9cb56177a511438f9f4a8c9ad5d64852, State: Running, Role: LEADER
I20260807 01:45:18.665520 25106 consensus_queue.cc:237] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.679257 24913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0fd88947c65482b8e9d6b23e96a4db2" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
W20260807 01:45:18.679013 24864 leader_election.cc:343] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099): Illegal state: must be running to vote when last-logged opid is not known
W20260807 01:45:18.687173 24742 leader_election.cc:343] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:18.689124 25026 raft_consensus.cc:2468] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f22c8ad8eba94330950285be7e2a09a8 in term 1.
I20260807 01:45:18.689834 25130 raft_consensus.cc:2749] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:45:18.690706 24861 leader_election.cc:304] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1, f22c8ad8eba94330950285be7e2a09a8; no voters: 9cb56177a511438f9f4a8c9ad5d64852
I20260807 01:45:18.693311 25142 raft_consensus.cc:2804] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:45:18.694789 25142 raft_consensus.cc:697] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 1 LEADER]: Becoming Leader. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Running, Role: LEADER
I20260807 01:45:18.697574 25142 consensus_queue.cc:237] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.694494 25094 raft_consensus.cc:359] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.700834 25094 raft_consensus.cc:385] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:18.701622 25094 raft_consensus.cc:740] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb56177a511438f9f4a8c9ad5d64852, State: Initialized, Role: FOLLOWER
I20260807 01:45:18.703285 25094 consensus_queue.cc:260] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.706758 24636 catalog_manager.cc:5697] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1). New cstate: current_term: 1 leader_uuid: "9cb56177a511438f9f4a8c9ad5d64852" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:18.712394 25096 tablet_bootstrap.cc:654] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:18.710737 24637 catalog_manager.cc:5697] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2). New cstate: current_term: 1 leader_uuid: "f22c8ad8eba94330950285be7e2a09a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:18.725575 25095 tablet_bootstrap.cc:492] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1: No bootstrap required, opened a new log
I20260807 01:45:18.726627 25095 ts_tablet_manager.cc:1403] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1: Time spent bootstrapping tablet: real 0.120s user 0.010s sys 0.030s
I20260807 01:45:18.727269 25107 raft_consensus.cc:493] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:18.728127 25107 raft_consensus.cc:515] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.733955 24800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b606cc1d46c3462f9759d92957b384f0" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852" is_pre_election: true
I20260807 01:45:18.737984 25026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b606cc1d46c3462f9759d92957b384f0" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a5248f4bf84ded832b7e261c3768b1" is_pre_election: true
W20260807 01:45:18.738405 24864 leader_election.cc:343] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:18.738907 25026 raft_consensus.cc:2468] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f22c8ad8eba94330950285be7e2a09a8 in term 0.
I20260807 01:45:18.735942 25107 leader_election.cc:290] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:18.739454 25095 raft_consensus.cc:359] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.742359 25095 raft_consensus.cc:385] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:18.743142 25095 raft_consensus.cc:740] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a5248f4bf84ded832b7e261c3768b1, State: Initialized, Role: FOLLOWER
I20260807 01:45:18.744383 25095 consensus_queue.cc:260] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.748158 25095 ts_tablet_manager.cc:1434] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1: Time spent starting tablet: real 0.021s user 0.013s sys 0.000s
I20260807 01:45:18.749954 25095 tablet_bootstrap.cc:492] T 719f845bd45941dcaa2421158f4b0ebe P 07a5248f4bf84ded832b7e261c3768b1: Bootstrap starting.
I20260807 01:45:18.759634 24861 leader_election.cc:304] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1, f22c8ad8eba94330950285be7e2a09a8; no voters: 9cb56177a511438f9f4a8c9ad5d64852
I20260807 01:45:18.764954 25094 ts_tablet_manager.cc:1434] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852: Time spent starting tablet: real 0.110s user 0.009s sys 0.009s
I20260807 01:45:18.764169 25107 raft_consensus.cc:2804] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:45:18.775683 25107 raft_consensus.cc:493] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:45:18.776468 25107 raft_consensus.cc:3060] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:18.785261 25094 tablet_bootstrap.cc:492] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852: Bootstrap starting.
I20260807 01:45:18.801283 25096 tablet_bootstrap.cc:492] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8: No bootstrap required, opened a new log
I20260807 01:45:18.806245 25107 raft_consensus.cc:515] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.808920 25096 ts_tablet_manager.cc:1403] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8: Time spent bootstrapping tablet: real 0.266s user 0.025s sys 0.023s
I20260807 01:45:18.814837 24800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b606cc1d46c3462f9759d92957b384f0" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
W20260807 01:45:18.817272 24864 leader_election.cc:343] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:18.822041 25095 tablet_bootstrap.cc:654] T 719f845bd45941dcaa2421158f4b0ebe P 07a5248f4bf84ded832b7e261c3768b1: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:18.821514 25026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b606cc1d46c3462f9759d92957b384f0" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
I20260807 01:45:18.820988 25107 leader_election.cc:290] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 election: Requested vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:18.822706 25026 raft_consensus.cc:3060] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:18.827226 25130 raft_consensus.cc:493] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:18.831686 25094 tablet_bootstrap.cc:654] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:18.828228 25130 raft_consensus.cc:515] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.833271 25026 raft_consensus.cc:2468] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f22c8ad8eba94330950285be7e2a09a8 in term 1.
I20260807 01:45:18.835258 25096 raft_consensus.cc:359] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.836506 25096 raft_consensus.cc:385] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:18.836910 25096 raft_consensus.cc:740] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Initialized, Role: FOLLOWER
I20260807 01:45:18.838043 25096 consensus_queue.cc:260] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.840909 24861 leader_election.cc:304] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1, f22c8ad8eba94330950285be7e2a09a8; no voters: 9cb56177a511438f9f4a8c9ad5d64852
I20260807 01:45:18.842265 25096 ts_tablet_manager.cc:1434] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8: Time spent starting tablet: real 0.033s user 0.007s sys 0.006s
I20260807 01:45:18.843124 25142 raft_consensus.cc:2804] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:45:18.843961 25096 tablet_bootstrap.cc:492] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8: Bootstrap starting.
I20260807 01:45:18.843837 25142 raft_consensus.cc:697] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [term 1 LEADER]: Becoming Leader. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Running, Role: LEADER
I20260807 01:45:18.845233 25142 consensus_queue.cc:237] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.849790 24800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "088984f6ac834d64b99408685df54428" candidate_uuid: "07a5248f4bf84ded832b7e261c3768b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852" is_pre_election: true
I20260807 01:45:18.851747 24800 raft_consensus.cc:2393] T 088984f6ac834d64b99408685df54428 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 07a5248f4bf84ded832b7e261c3768b1 in current term 1: Already voted for candidate f22c8ad8eba94330950285be7e2a09a8 in this term.
I20260807 01:45:18.855168 24913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "088984f6ac834d64b99408685df54428" candidate_uuid: "07a5248f4bf84ded832b7e261c3768b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f22c8ad8eba94330950285be7e2a09a8" is_pre_election: true
I20260807 01:45:18.869154 25096 tablet_bootstrap.cc:654] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:18.872668 25130 leader_election.cc:290] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449)
I20260807 01:45:18.872879 24974 leader_election.cc:304] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1; no voters: 9cb56177a511438f9f4a8c9ad5d64852, f22c8ad8eba94330950285be7e2a09a8
I20260807 01:45:18.875804 25130 raft_consensus.cc:3060] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:18.890990 25106 raft_consensus.cc:493] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:18.891579 25096 tablet_bootstrap.cc:492] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8: No bootstrap required, opened a new log
I20260807 01:45:18.892169 25096 ts_tablet_manager.cc:1403] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8: Time spent bootstrapping tablet: real 0.049s user 0.016s sys 0.014s
I20260807 01:45:18.891749 25106 raft_consensus.cc:515] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.898010 24913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719f845bd45941dcaa2421158f4b0ebe" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f22c8ad8eba94330950285be7e2a09a8" is_pre_election: true
I20260807 01:45:18.898818 25094 tablet_bootstrap.cc:492] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852: No bootstrap required, opened a new log
I20260807 01:45:18.898764 24913 raft_consensus.cc:2468] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb56177a511438f9f4a8c9ad5d64852 in term 0.
I20260807 01:45:18.900167 25026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719f845bd45941dcaa2421158f4b0ebe" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a5248f4bf84ded832b7e261c3768b1" is_pre_election: true
I20260807 01:45:18.900929 24742 leader_election.cc:304] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9cb56177a511438f9f4a8c9ad5d64852, f22c8ad8eba94330950285be7e2a09a8; no voters:
I20260807 01:45:18.900847 25096 raft_consensus.cc:359] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.902355 25096 raft_consensus.cc:385] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260807 01:45:18.902361 24738 leader_election.cc:343] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:18.902590 25137 raft_consensus.cc:2804] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:45:18.902872 25094 ts_tablet_manager.cc:1403] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852: Time spent bootstrapping tablet: real 0.118s user 0.023s sys 0.026s
I20260807 01:45:18.903040 25096 raft_consensus.cc:740] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Initialized, Role: FOLLOWER
I20260807 01:45:18.903421 25137 raft_consensus.cc:493] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:45:18.904186 25137 raft_consensus.cc:3060] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:18.904899 25096 consensus_queue.cc:260] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.909452 25096 ts_tablet_manager.cc:1434] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8: Time spent starting tablet: real 0.017s user 0.009s sys 0.005s
I20260807 01:45:18.896836 25106 leader_election.cc:290] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:18.911243 25096 tablet_bootstrap.cc:492] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8: Bootstrap starting.
I20260807 01:45:18.912833 25094 raft_consensus.cc:359] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.913771 25094 raft_consensus.cc:385] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:18.914204 25094 raft_consensus.cc:740] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb56177a511438f9f4a8c9ad5d64852, State: Initialized, Role: FOLLOWER
I20260807 01:45:18.915380 25094 consensus_queue.cc:260] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.913233 25095 tablet_bootstrap.cc:492] T 719f845bd45941dcaa2421158f4b0ebe P 07a5248f4bf84ded832b7e261c3768b1: No bootstrap required, opened a new log
I20260807 01:45:18.922529 25095 ts_tablet_manager.cc:1403] T 719f845bd45941dcaa2421158f4b0ebe P 07a5248f4bf84ded832b7e261c3768b1: Time spent bootstrapping tablet: real 0.173s user 0.019s sys 0.014s
I20260807 01:45:18.924500 25096 tablet_bootstrap.cc:654] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:18.925297 25130 raft_consensus.cc:2749] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:45:18.929435 25094 ts_tablet_manager.cc:1434] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852: Time spent starting tablet: real 0.022s user 0.006s sys 0.006s
I20260807 01:45:18.931643 25094 tablet_bootstrap.cc:492] T b606cc1d46c3462f9759d92957b384f0 P 9cb56177a511438f9f4a8c9ad5d64852: Bootstrap starting.
I20260807 01:45:18.932031 25095 raft_consensus.cc:359] T 719f845bd45941dcaa2421158f4b0ebe P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.933071 25095 raft_consensus.cc:385] T 719f845bd45941dcaa2421158f4b0ebe P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:18.935949 25137 raft_consensus.cc:515] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.940295 24913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719f845bd45941dcaa2421158f4b0ebe" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
I20260807 01:45:18.941033 24913 raft_consensus.cc:3060] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:18.933655 25095 raft_consensus.cc:740] T 719f845bd45941dcaa2421158f4b0ebe P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a5248f4bf84ded832b7e261c3768b1, State: Initialized, Role: FOLLOWER
I20260807 01:45:18.966416 25094 tablet_bootstrap.cc:654] T b606cc1d46c3462f9759d92957b384f0 P 9cb56177a511438f9f4a8c9ad5d64852: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:18.974295 25095 consensus_queue.cc:260] T 719f845bd45941dcaa2421158f4b0ebe P 07a5248f4bf84ded832b7e261c3768b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:18.978206 25095 ts_tablet_manager.cc:1434] T 719f845bd45941dcaa2421158f4b0ebe P 07a5248f4bf84ded832b7e261c3768b1: Time spent starting tablet: real 0.055s user 0.008s sys 0.004s
I20260807 01:45:18.984844 25137 leader_election.cc:290] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 1 election: Requested vote from peers f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:18.979590 25095 tablet_bootstrap.cc:492] T d3c14e19345c4777a5f505485d89b588 P 07a5248f4bf84ded832b7e261c3768b1: Bootstrap starting.
W20260807 01:45:18.982456 24708 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 01:45:18.986335 25026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719f845bd45941dcaa2421158f4b0ebe" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
I20260807 01:45:18.987159 25026 raft_consensus.cc:3060] T 719f845bd45941dcaa2421158f4b0ebe P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:18.985669 24913 raft_consensus.cc:2468] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb56177a511438f9f4a8c9ad5d64852 in term 1.
I20260807 01:45:18.990310 24742 leader_election.cc:304] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9cb56177a511438f9f4a8c9ad5d64852, f22c8ad8eba94330950285be7e2a09a8; no voters:
I20260807 01:45:18.991394 25137 raft_consensus.cc:2804] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:45:18.992671 25137 raft_consensus.cc:697] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 LEADER]: Becoming Leader. State: Replica: 9cb56177a511438f9f4a8c9ad5d64852, State: Running, Role: LEADER
I20260807 01:45:18.994867 25137 consensus_queue.cc:237] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.001819 25096 tablet_bootstrap.cc:492] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8: No bootstrap required, opened a new log
I20260807 01:45:19.002624 25096 ts_tablet_manager.cc:1403] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8: Time spent bootstrapping tablet: real 0.092s user 0.032s sys 0.027s
I20260807 01:45:19.011790 25106 consensus_queue.cc:1048] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:45:19.013245 25026 raft_consensus.cc:2468] T 719f845bd45941dcaa2421158f4b0ebe P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb56177a511438f9f4a8c9ad5d64852 in term 1.
I20260807 01:45:19.012470 24636 catalog_manager.cc:5697] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2). New cstate: current_term: 1 leader_uuid: "f22c8ad8eba94330950285be7e2a09a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:19.017184 24636 catalog_manager.cc:5697] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2). New cstate: current_term: 1 leader_uuid: "f22c8ad8eba94330950285be7e2a09a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:19.018381 25095 tablet_bootstrap.cc:654] T d3c14e19345c4777a5f505485d89b588 P 07a5248f4bf84ded832b7e261c3768b1: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:19.038350 25095 tablet_bootstrap.cc:492] T d3c14e19345c4777a5f505485d89b588 P 07a5248f4bf84ded832b7e261c3768b1: No bootstrap required, opened a new log
I20260807 01:45:19.039288 25095 ts_tablet_manager.cc:1403] T d3c14e19345c4777a5f505485d89b588 P 07a5248f4bf84ded832b7e261c3768b1: Time spent bootstrapping tablet: real 0.060s user 0.024s sys 0.004s
I20260807 01:45:19.032995 25096 raft_consensus.cc:359] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.042465 25096 raft_consensus.cc:385] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:19.042881 25096 raft_consensus.cc:740] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Initialized, Role: FOLLOWER
I20260807 01:45:19.044181 25096 consensus_queue.cc:260] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.044250 24637 catalog_manager.cc:5697] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1). New cstate: current_term: 1 leader_uuid: "9cb56177a511438f9f4a8c9ad5d64852" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:19.049070 25095 raft_consensus.cc:359] T d3c14e19345c4777a5f505485d89b588 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.052428 25095 raft_consensus.cc:385] T d3c14e19345c4777a5f505485d89b588 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:19.053129 25095 raft_consensus.cc:740] T d3c14e19345c4777a5f505485d89b588 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a5248f4bf84ded832b7e261c3768b1, State: Initialized, Role: FOLLOWER
I20260807 01:45:19.049101 25096 ts_tablet_manager.cc:1434] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8: Time spent starting tablet: real 0.046s user 0.005s sys 0.008s
I20260807 01:45:19.054733 25095 consensus_queue.cc:260] T d3c14e19345c4777a5f505485d89b588 P 07a5248f4bf84ded832b7e261c3768b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.059469 25096 tablet_bootstrap.cc:492] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8: Bootstrap starting.
I20260807 01:45:19.062248 25095 ts_tablet_manager.cc:1434] T d3c14e19345c4777a5f505485d89b588 P 07a5248f4bf84ded832b7e261c3768b1: Time spent starting tablet: real 0.022s user 0.004s sys 0.012s
I20260807 01:45:19.064273 25095 tablet_bootstrap.cc:492] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1: Bootstrap starting.
I20260807 01:45:19.068272 25142 raft_consensus.cc:493] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:19.068931 25142 raft_consensus.cc:515] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.073457 25107 raft_consensus.cc:493] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:19.074743 25025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c14e19345c4777a5f505485d89b588" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a5248f4bf84ded832b7e261c3768b1" is_pre_election: true
I20260807 01:45:19.074473 25107 raft_consensus.cc:515] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.075484 25025 raft_consensus.cc:2468] T d3c14e19345c4777a5f505485d89b588 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f22c8ad8eba94330950285be7e2a09a8 in term 0.
I20260807 01:45:19.077702 25142 leader_election.cc:290] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:19.077337 24861 leader_election.cc:304] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1, f22c8ad8eba94330950285be7e2a09a8; no voters:
I20260807 01:45:19.080680 25025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0fd88947c65482b8e9d6b23e96a4db2" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a5248f4bf84ded832b7e261c3768b1" is_pre_election: true
I20260807 01:45:19.081367 24800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c14e19345c4777a5f505485d89b588" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852" is_pre_election: true
I20260807 01:45:19.081815 25025 raft_consensus.cc:2393] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f22c8ad8eba94330950285be7e2a09a8 in current term 1: Already voted for candidate 9cb56177a511438f9f4a8c9ad5d64852 in this term.
I20260807 01:45:19.077482 25107 leader_election.cc:290] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:19.082789 25107 raft_consensus.cc:2804] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:45:19.082973 24795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0fd88947c65482b8e9d6b23e96a4db2" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852" is_pre_election: true
W20260807 01:45:19.088258 24864 leader_election.cc:343] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:19.089440 24864 leader_election.cc:304] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f22c8ad8eba94330950285be7e2a09a8; no voters: 07a5248f4bf84ded832b7e261c3768b1, 9cb56177a511438f9f4a8c9ad5d64852
I20260807 01:45:19.090401 25142 raft_consensus.cc:3060] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:19.091557 25095 tablet_bootstrap.cc:654] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:19.083305 25107 raft_consensus.cc:493] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:45:19.094669 25107 raft_consensus.cc:3060] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:19.106575 25142 raft_consensus.cc:2749] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:45:19.111490 25094 tablet_bootstrap.cc:492] T b606cc1d46c3462f9759d92957b384f0 P 9cb56177a511438f9f4a8c9ad5d64852: No bootstrap required, opened a new log
I20260807 01:45:19.111579 25096 tablet_bootstrap.cc:654] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:19.112489 25094 ts_tablet_manager.cc:1403] T b606cc1d46c3462f9759d92957b384f0 P 9cb56177a511438f9f4a8c9ad5d64852: Time spent bootstrapping tablet: real 0.181s user 0.007s sys 0.035s
I20260807 01:45:19.131965 25107 raft_consensus.cc:515] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.132570 24636 catalog_manager.cc:5697] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1). New cstate: current_term: 1 leader_uuid: "9cb56177a511438f9f4a8c9ad5d64852" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:19.141598 25094 raft_consensus.cc:359] T b606cc1d46c3462f9759d92957b384f0 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.146490 25094 raft_consensus.cc:385] T b606cc1d46c3462f9759d92957b384f0 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:19.147217 25094 raft_consensus.cc:740] T b606cc1d46c3462f9759d92957b384f0 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb56177a511438f9f4a8c9ad5d64852, State: Initialized, Role: FOLLOWER
I20260807 01:45:19.148425 25094 consensus_queue.cc:260] T b606cc1d46c3462f9759d92957b384f0 P 9cb56177a511438f9f4a8c9ad5d64852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.159047 25137 consensus_queue.cc:1048] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:45:19.168903 25142 consensus_queue.cc:1048] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:45:19.191979 25024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c14e19345c4777a5f505485d89b588" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
I20260807 01:45:19.182691 25107 leader_election.cc:290] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 election: Requested vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:19.198248 25096 tablet_bootstrap.cc:492] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8: No bootstrap required, opened a new log
I20260807 01:45:19.207618 25095 tablet_bootstrap.cc:492] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1: No bootstrap required, opened a new log
I20260807 01:45:19.207775 25096 ts_tablet_manager.cc:1403] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8: Time spent bootstrapping tablet: real 0.149s user 0.024s sys 0.011s
I20260807 01:45:19.208215 25095 ts_tablet_manager.cc:1403] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1: Time spent bootstrapping tablet: real 0.144s user 0.017s sys 0.024s
I20260807 01:45:19.240696 25095 raft_consensus.cc:359] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.241533 25095 raft_consensus.cc:385] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:19.241806 25095 raft_consensus.cc:740] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a5248f4bf84ded832b7e261c3768b1, State: Initialized, Role: FOLLOWER
I20260807 01:45:19.242566 25095 consensus_queue.cc:260] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.244244 25095 ts_tablet_manager.cc:1434] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1: Time spent starting tablet: real 0.036s user 0.003s sys 0.003s
I20260807 01:45:19.245278 25095 tablet_bootstrap.cc:492] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1: Bootstrap starting.
I20260807 01:45:19.209128 25024 raft_consensus.cc:3060] T d3c14e19345c4777a5f505485d89b588 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:19.264896 25106 consensus_queue.cc:1048] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260807 01:45:19.269088 25096 raft_consensus.cc:359] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.269802 25096 raft_consensus.cc:385] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:19.270133 25096 raft_consensus.cc:740] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Initialized, Role: FOLLOWER
I20260807 01:45:19.261992 24795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c14e19345c4777a5f505485d89b588" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
I20260807 01:45:19.274442 25096 consensus_queue.cc:260] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.277415 24792 raft_consensus.cc:3060] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:19.285336 25096 ts_tablet_manager.cc:1434] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8: Time spent starting tablet: real 0.077s user 0.002s sys 0.006s
W20260807 01:45:19.291375 24864 consensus_peers.cc:597] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 -> Peer 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099): Couldn't send request to peer 9cb56177a511438f9f4a8c9ad5d64852. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260807 01:45:19.292510 24864 leader_election.cc:343] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:19.292630 25107 consensus_queue.cc:1048] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:45:19.296258 25096 tablet_bootstrap.cc:492] T 78946d0ad61f46a7a81f1592fa57a8d3 P f22c8ad8eba94330950285be7e2a09a8: Bootstrap starting.
I20260807 01:45:19.300804 25198 consensus_queue.cc:1048] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260807 01:45:19.318452 25137 consensus_queue.cc:1048] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260807 01:45:19.355086 25024 raft_consensus.cc:2468] T d3c14e19345c4777a5f505485d89b588 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f22c8ad8eba94330950285be7e2a09a8 in term 1.
I20260807 01:45:19.358143 24861 leader_election.cc:304] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1, f22c8ad8eba94330950285be7e2a09a8; no voters: 9cb56177a511438f9f4a8c9ad5d64852
I20260807 01:45:19.363004 25142 raft_consensus.cc:2804] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:45:19.372819 25106 raft_consensus.cc:493] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:19.373616 25106 raft_consensus.cc:515] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.385430 25137 raft_consensus.cc:493] T b606cc1d46c3462f9759d92957b384f0 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:19.386132 25137 raft_consensus.cc:515] T b606cc1d46c3462f9759d92957b384f0 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.398983 25137 leader_election.cc:290] T b606cc1d46c3462f9759d92957b384f0 P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:19.400260 24911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b606cc1d46c3462f9759d92957b384f0" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f22c8ad8eba94330950285be7e2a09a8" is_pre_election: true
I20260807 01:45:19.413921 25022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b606cc1d46c3462f9759d92957b384f0" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a5248f4bf84ded832b7e261c3768b1" is_pre_election: true
I20260807 01:45:19.415472 25022 raft_consensus.cc:2393] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9cb56177a511438f9f4a8c9ad5d64852 in current term 1: Already voted for candidate f22c8ad8eba94330950285be7e2a09a8 in this term.
I20260807 01:45:19.415799 25095 tablet_bootstrap.cc:654] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:19.418057 24738 leader_election.cc:304] T b606cc1d46c3462f9759d92957b384f0 P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9cb56177a511438f9f4a8c9ad5d64852; no voters: 07a5248f4bf84ded832b7e261c3768b1, f22c8ad8eba94330950285be7e2a09a8
I20260807 01:45:19.398485 25198 consensus_queue.cc:1048] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260807 01:45:19.421474 25194 raft_consensus.cc:3060] T b606cc1d46c3462f9759d92957b384f0 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:19.440862 25130 raft_consensus.cc:493] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:19.442086 25130 raft_consensus.cc:515] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.472841 25094 ts_tablet_manager.cc:1434] T b606cc1d46c3462f9759d92957b384f0 P 9cb56177a511438f9f4a8c9ad5d64852: Time spent starting tablet: real 0.360s user 0.021s sys 0.033s
I20260807 01:45:19.496886 25194 raft_consensus.cc:2749] T b606cc1d46c3462f9759d92957b384f0 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:45:19.499457 25130 leader_election.cc:290] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449)
I20260807 01:45:19.502266 24911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc90e28bc0a843b1a8485dafefcd4a05" candidate_uuid: "07a5248f4bf84ded832b7e261c3768b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f22c8ad8eba94330950285be7e2a09a8" is_pre_election: true
I20260807 01:45:19.506549 24911 raft_consensus.cc:2468] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07a5248f4bf84ded832b7e261c3768b1 in term 0.
I20260807 01:45:19.505273 25021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2791c7f6f9c647df909f735bd72c5a63" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a5248f4bf84ded832b7e261c3768b1" is_pre_election: true
I20260807 01:45:19.502933 25106 leader_election.cc:290] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:19.518710 24974 leader_election.cc:304] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1, f22c8ad8eba94330950285be7e2a09a8; no voters:
I20260807 01:45:19.519922 25130 raft_consensus.cc:2804] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:45:19.520588 25130 raft_consensus.cc:493] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:45:19.521201 25130 raft_consensus.cc:3060] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:19.521808 25094 tablet_bootstrap.cc:492] T d3c14e19345c4777a5f505485d89b588 P 9cb56177a511438f9f4a8c9ad5d64852: Bootstrap starting.
I20260807 01:45:19.522644 24795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc90e28bc0a843b1a8485dafefcd4a05" candidate_uuid: "07a5248f4bf84ded832b7e261c3768b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852" is_pre_election: true
W20260807 01:45:19.526109 24973 leader_election.cc:343] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:19.528271 24911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2791c7f6f9c647df909f735bd72c5a63" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f22c8ad8eba94330950285be7e2a09a8" is_pre_election: true
I20260807 01:45:19.535766 25096 tablet_bootstrap.cc:654] T 78946d0ad61f46a7a81f1592fa57a8d3 P f22c8ad8eba94330950285be7e2a09a8: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:19.528960 25142 raft_consensus.cc:697] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 [term 1 LEADER]: Becoming Leader. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Running, Role: LEADER
I20260807 01:45:19.548936 25198 consensus_queue.cc:1048] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260807 01:45:19.571524 24742 leader_election.cc:304] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9cb56177a511438f9f4a8c9ad5d64852; no voters: 07a5248f4bf84ded832b7e261c3768b1, f22c8ad8eba94330950285be7e2a09a8
I20260807 01:45:19.554745 25142 consensus_queue.cc:237] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.571311 25204 consensus_queue.cc:1048] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260807 01:45:19.578390 25106 raft_consensus.cc:2749] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:45:19.580569 25137 consensus_queue.cc:1048] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:45:19.605058 25130 raft_consensus.cc:515] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.629158 25130 leader_election.cc:290] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 1 election: Requested vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449)
I20260807 01:45:19.631840 24912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc90e28bc0a843b1a8485dafefcd4a05" candidate_uuid: "07a5248f4bf84ded832b7e261c3768b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
I20260807 01:45:19.631742 25094 tablet_bootstrap.cc:654] T d3c14e19345c4777a5f505485d89b588 P 9cb56177a511438f9f4a8c9ad5d64852: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:19.634349 24912 raft_consensus.cc:3060] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:19.636376 24792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc90e28bc0a843b1a8485dafefcd4a05" candidate_uuid: "07a5248f4bf84ded832b7e261c3768b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
I20260807 01:45:19.663398 25095 tablet_bootstrap.cc:492] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1: No bootstrap required, opened a new log
I20260807 01:45:19.664017 25095 ts_tablet_manager.cc:1403] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1: Time spent bootstrapping tablet: real 0.419s user 0.012s sys 0.017s
W20260807 01:45:19.670928 24973 leader_election.cc:343] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:19.683061 25095 raft_consensus.cc:359] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.683980 25095 raft_consensus.cc:385] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:19.684433 25095 raft_consensus.cc:740] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a5248f4bf84ded832b7e261c3768b1, State: Initialized, Role: FOLLOWER
I20260807 01:45:19.685716 25095 consensus_queue.cc:260] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.690367 25095 ts_tablet_manager.cc:1434] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1: Time spent starting tablet: real 0.026s user 0.004s sys 0.006s
I20260807 01:45:19.696017 24912 raft_consensus.cc:2468] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a5248f4bf84ded832b7e261c3768b1 in term 1.
I20260807 01:45:19.704186 24974 leader_election.cc:304] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1, f22c8ad8eba94330950285be7e2a09a8; no voters: 9cb56177a511438f9f4a8c9ad5d64852
I20260807 01:45:19.709950 25216 raft_consensus.cc:2804] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:45:19.713528 25216 raft_consensus.cc:697] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 LEADER]: Becoming Leader. State: Replica: 07a5248f4bf84ded832b7e261c3768b1, State: Running, Role: LEADER
I20260807 01:45:19.720311 25216 consensus_queue.cc:237] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.751971 25094 tablet_bootstrap.cc:492] T d3c14e19345c4777a5f505485d89b588 P 9cb56177a511438f9f4a8c9ad5d64852: No bootstrap required, opened a new log
I20260807 01:45:19.766290 25096 tablet_bootstrap.cc:492] T 78946d0ad61f46a7a81f1592fa57a8d3 P f22c8ad8eba94330950285be7e2a09a8: No bootstrap required, opened a new log
I20260807 01:45:19.775779 25094 ts_tablet_manager.cc:1403] T d3c14e19345c4777a5f505485d89b588 P 9cb56177a511438f9f4a8c9ad5d64852: Time spent bootstrapping tablet: real 0.254s user 0.011s sys 0.023s
I20260807 01:45:19.789686 25096 ts_tablet_manager.cc:1403] T 78946d0ad61f46a7a81f1592fa57a8d3 P f22c8ad8eba94330950285be7e2a09a8: Time spent bootstrapping tablet: real 0.494s user 0.017s sys 0.033s
I20260807 01:45:19.781704 24637 catalog_manager.cc:5697] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2). New cstate: current_term: 1 leader_uuid: "f22c8ad8eba94330950285be7e2a09a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:19.797508 25096 raft_consensus.cc:359] T 78946d0ad61f46a7a81f1592fa57a8d3 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.800645 25094 raft_consensus.cc:359] T d3c14e19345c4777a5f505485d89b588 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.801909 25096 raft_consensus.cc:385] T 78946d0ad61f46a7a81f1592fa57a8d3 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:19.802338 25096 raft_consensus.cc:740] T 78946d0ad61f46a7a81f1592fa57a8d3 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Initialized, Role: FOLLOWER
I20260807 01:45:19.802286 25094 raft_consensus.cc:385] T d3c14e19345c4777a5f505485d89b588 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:19.803117 25094 raft_consensus.cc:740] T d3c14e19345c4777a5f505485d89b588 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb56177a511438f9f4a8c9ad5d64852, State: Initialized, Role: FOLLOWER
I20260807 01:45:19.803740 25096 consensus_queue.cc:260] T 78946d0ad61f46a7a81f1592fa57a8d3 P f22c8ad8eba94330950285be7e2a09a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.804302 25094 consensus_queue.cc:260] T d3c14e19345c4777a5f505485d89b588 P 9cb56177a511438f9f4a8c9ad5d64852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.806607 25096 ts_tablet_manager.cc:1434] T 78946d0ad61f46a7a81f1592fa57a8d3 P f22c8ad8eba94330950285be7e2a09a8: Time spent starting tablet: real 0.016s user 0.007s sys 0.001s
I20260807 01:45:19.808077 25216 raft_consensus.cc:493] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:19.808861 25216 raft_consensus.cc:515] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.811776 25094 ts_tablet_manager.cc:1434] T d3c14e19345c4777a5f505485d89b588 P 9cb56177a511438f9f4a8c9ad5d64852: Time spent starting tablet: real 0.022s user 0.005s sys 0.007s
I20260807 01:45:19.813611 25094 tablet_bootstrap.cc:492] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852: Bootstrap starting.
I20260807 01:45:19.815024 24792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78946d0ad61f46a7a81f1592fa57a8d3" candidate_uuid: "07a5248f4bf84ded832b7e261c3768b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852" is_pre_election: true
W20260807 01:45:19.818104 24973 leader_election.cc:343] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:19.820968 25216 leader_election.cc:290] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449)
I20260807 01:45:19.822618 24911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78946d0ad61f46a7a81f1592fa57a8d3" candidate_uuid: "07a5248f4bf84ded832b7e261c3768b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f22c8ad8eba94330950285be7e2a09a8" is_pre_election: true
I20260807 01:45:19.823496 24911 raft_consensus.cc:2468] T 78946d0ad61f46a7a81f1592fa57a8d3 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07a5248f4bf84ded832b7e261c3768b1 in term 0.
I20260807 01:45:19.825668 24974 leader_election.cc:304] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1, f22c8ad8eba94330950285be7e2a09a8; no voters: 9cb56177a511438f9f4a8c9ad5d64852
I20260807 01:45:19.827955 25216 raft_consensus.cc:2804] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:45:19.828898 25216 raft_consensus.cc:493] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:45:19.829571 25216 raft_consensus.cc:3060] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:19.830294 25094 tablet_bootstrap.cc:654] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:19.850967 25216 raft_consensus.cc:515] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.852752 24636 catalog_manager.cc:5697] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3). New cstate: current_term: 1 leader_uuid: "07a5248f4bf84ded832b7e261c3768b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } health_report { overall_health: HEALTHY } } }
I20260807 01:45:19.857506 24800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78946d0ad61f46a7a81f1592fa57a8d3" candidate_uuid: "07a5248f4bf84ded832b7e261c3768b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
I20260807 01:45:19.858480 25194 raft_consensus.cc:493] T d3c14e19345c4777a5f505485d89b588 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:19.859138 25194 raft_consensus.cc:515] T d3c14e19345c4777a5f505485d89b588 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
W20260807 01:45:19.859841 24973 leader_election.cc:343] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:19.864068 24911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78946d0ad61f46a7a81f1592fa57a8d3" candidate_uuid: "07a5248f4bf84ded832b7e261c3768b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
I20260807 01:45:19.864995 24911 raft_consensus.cc:3060] T 78946d0ad61f46a7a81f1592fa57a8d3 P f22c8ad8eba94330950285be7e2a09a8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:19.865547 25022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c14e19345c4777a5f505485d89b588" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a5248f4bf84ded832b7e261c3768b1" is_pre_election: true
I20260807 01:45:19.866655 25022 raft_consensus.cc:2393] T d3c14e19345c4777a5f505485d89b588 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9cb56177a511438f9f4a8c9ad5d64852 in current term 1: Already voted for candidate f22c8ad8eba94330950285be7e2a09a8 in this term.
I20260807 01:45:19.868628 25194 leader_election.cc:290] T d3c14e19345c4777a5f505485d89b588 P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:19.870437 24912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c14e19345c4777a5f505485d89b588" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f22c8ad8eba94330950285be7e2a09a8" is_pre_election: true
I20260807 01:45:19.876955 25094 tablet_bootstrap.cc:492] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852: No bootstrap required, opened a new log
I20260807 01:45:19.876479 24742 leader_election.cc:304] T d3c14e19345c4777a5f505485d89b588 P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9cb56177a511438f9f4a8c9ad5d64852; no voters: 07a5248f4bf84ded832b7e261c3768b1, f22c8ad8eba94330950285be7e2a09a8
I20260807 01:45:19.878013 25094 ts_tablet_manager.cc:1403] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852: Time spent bootstrapping tablet: real 0.065s user 0.025s sys 0.029s
I20260807 01:45:19.878342 25194 raft_consensus.cc:3060] T d3c14e19345c4777a5f505485d89b588 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:19.870464 25216 leader_election.cc:290] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 1 election: Requested vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449)
I20260807 01:45:19.882001 24911 raft_consensus.cc:2468] T 78946d0ad61f46a7a81f1592fa57a8d3 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a5248f4bf84ded832b7e261c3768b1 in term 1.
I20260807 01:45:19.883801 24974 leader_election.cc:304] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1, f22c8ad8eba94330950285be7e2a09a8; no voters: 9cb56177a511438f9f4a8c9ad5d64852
I20260807 01:45:19.884035 25094 raft_consensus.cc:359] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.884923 25094 raft_consensus.cc:385] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:19.884722 25216 raft_consensus.cc:2804] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:45:19.885291 25094 raft_consensus.cc:740] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb56177a511438f9f4a8c9ad5d64852, State: Initialized, Role: FOLLOWER
I20260807 01:45:19.886093 25216 raft_consensus.cc:697] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 LEADER]: Becoming Leader. State: Replica: 07a5248f4bf84ded832b7e261c3768b1, State: Running, Role: LEADER
I20260807 01:45:19.886245 25094 consensus_queue.cc:260] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.888756 25094 ts_tablet_manager.cc:1434] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260807 01:45:19.888321 25216 consensus_queue.cc:237] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.890630 25094 tablet_bootstrap.cc:492] T 78946d0ad61f46a7a81f1592fa57a8d3 P 9cb56177a511438f9f4a8c9ad5d64852: Bootstrap starting.
I20260807 01:45:19.897081 25194 raft_consensus.cc:2749] T d3c14e19345c4777a5f505485d89b588 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:45:19.905889 25094 tablet_bootstrap.cc:654] T 78946d0ad61f46a7a81f1592fa57a8d3 P 9cb56177a511438f9f4a8c9ad5d64852: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:19.929128 25194 raft_consensus.cc:493] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:19.930356 25194 raft_consensus.cc:515] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.935509 24911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc90e28bc0a843b1a8485dafefcd4a05" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f22c8ad8eba94330950285be7e2a09a8" is_pre_election: true
I20260807 01:45:19.936766 24911 raft_consensus.cc:2393] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9cb56177a511438f9f4a8c9ad5d64852 in current term 1: Already voted for candidate 07a5248f4bf84ded832b7e261c3768b1 in this term.
I20260807 01:45:19.936357 25022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc90e28bc0a843b1a8485dafefcd4a05" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a5248f4bf84ded832b7e261c3768b1" is_pre_election: true
I20260807 01:45:19.938086 25194 leader_election.cc:290] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:19.940112 24742 leader_election.cc:304] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9cb56177a511438f9f4a8c9ad5d64852; no voters: 07a5248f4bf84ded832b7e261c3768b1, f22c8ad8eba94330950285be7e2a09a8
I20260807 01:45:19.941627 25194 raft_consensus.cc:3060] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:19.944149 25094 tablet_bootstrap.cc:492] T 78946d0ad61f46a7a81f1592fa57a8d3 P 9cb56177a511438f9f4a8c9ad5d64852: No bootstrap required, opened a new log
I20260807 01:45:19.944808 25094 ts_tablet_manager.cc:1403] T 78946d0ad61f46a7a81f1592fa57a8d3 P 9cb56177a511438f9f4a8c9ad5d64852: Time spent bootstrapping tablet: real 0.055s user 0.004s sys 0.040s
I20260807 01:45:19.948741 25094 raft_consensus.cc:359] T 78946d0ad61f46a7a81f1592fa57a8d3 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.949587 25094 raft_consensus.cc:385] T 78946d0ad61f46a7a81f1592fa57a8d3 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:19.949928 25094 raft_consensus.cc:740] T 78946d0ad61f46a7a81f1592fa57a8d3 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb56177a511438f9f4a8c9ad5d64852, State: Initialized, Role: FOLLOWER
I20260807 01:45:19.950861 25094 consensus_queue.cc:260] T 78946d0ad61f46a7a81f1592fa57a8d3 P 9cb56177a511438f9f4a8c9ad5d64852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:19.953807 25194 raft_consensus.cc:2749] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:45:19.955704 25094 ts_tablet_manager.cc:1434] T 78946d0ad61f46a7a81f1592fa57a8d3 P 9cb56177a511438f9f4a8c9ad5d64852: Time spent starting tablet: real 0.010s user 0.005s sys 0.003s
I20260807 01:45:19.966213 24639 catalog_manager.cc:5697] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3). New cstate: current_term: 1 leader_uuid: "07a5248f4bf84ded832b7e261c3768b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } health_report { overall_health: HEALTHY } } }
I20260807 01:45:20.055246 25205 consensus_queue.cc:1048] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:45:20.085145 25214 consensus_queue.cc:1048] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:45:20.091476 24911 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2791c7f6f9c647df909f735bd72c5a63"
dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
mode: GRACEFUL
new_leader_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
from {username='slave'} at 127.0.0.1:45656
I20260807 01:45:20.092521 24911 raft_consensus.cc:606] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 1 LEADER]: Received request to transfer leadership to TS 9cb56177a511438f9f4a8c9ad5d64852
I20260807 01:45:20.116696 25194 raft_consensus.cc:493] T 78946d0ad61f46a7a81f1592fa57a8d3 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:20.117704 25194 raft_consensus.cc:515] T 78946d0ad61f46a7a81f1592fa57a8d3 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:20.120407 25194 leader_election.cc:290] T 78946d0ad61f46a7a81f1592fa57a8d3 P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:20.122275 24911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78946d0ad61f46a7a81f1592fa57a8d3" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f22c8ad8eba94330950285be7e2a09a8" is_pre_election: true
I20260807 01:45:20.124018 24911 raft_consensus.cc:2393] T 78946d0ad61f46a7a81f1592fa57a8d3 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9cb56177a511438f9f4a8c9ad5d64852 in current term 1: Already voted for candidate 07a5248f4bf84ded832b7e261c3768b1 in this term.
I20260807 01:45:20.127099 25022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78946d0ad61f46a7a81f1592fa57a8d3" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a5248f4bf84ded832b7e261c3768b1" is_pre_election: true
I20260807 01:45:20.128935 24738 leader_election.cc:304] T 78946d0ad61f46a7a81f1592fa57a8d3 P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9cb56177a511438f9f4a8c9ad5d64852; no voters: 07a5248f4bf84ded832b7e261c3768b1, f22c8ad8eba94330950285be7e2a09a8
I20260807 01:45:20.130368 25194 raft_consensus.cc:3060] T 78946d0ad61f46a7a81f1592fa57a8d3 P 9cb56177a511438f9f4a8c9ad5d64852 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:20.143754 25194 raft_consensus.cc:2749] T 78946d0ad61f46a7a81f1592fa57a8d3 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:45:20.157178 25022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc90e28bc0a843b1a8485dafefcd4a05"
dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
mode: GRACEFUL
new_leader_uuid: "f22c8ad8eba94330950285be7e2a09a8"
from {username='slave'} at 127.0.0.1:55850
I20260807 01:45:20.158385 25022 raft_consensus.cc:606] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 LEADER]: Received request to transfer leadership to TS f22c8ad8eba94330950285be7e2a09a8
I20260807 01:45:20.233590 25216 consensus_queue.cc:1048] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:45:20.246205 25221 raft_consensus.cc:993] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1: : Instructing follower f22c8ad8eba94330950285be7e2a09a8 to start an election
I20260807 01:45:20.247417 25216 raft_consensus.cc:1081] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 LEADER]: Signalling peer f22c8ad8eba94330950285be7e2a09a8 to start an election
I20260807 01:45:20.252738 24912 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dc90e28bc0a843b1a8485dafefcd4a05"
dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
from {username='slave'} at 127.0.0.1:45652
I20260807 01:45:20.253793 24912 raft_consensus.cc:493] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:45:20.254292 24912 raft_consensus.cc:3060] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:20.257476 25216 consensus_queue.cc:1048] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:45:20.269475 24912 raft_consensus.cc:515] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:20.272730 24912 leader_election.cc:290] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 2 election: Requested vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:20.273830 24792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc90e28bc0a843b1a8485dafefcd4a05" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
I20260807 01:45:20.274544 24792 raft_consensus.cc:3060] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:20.278053 25024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc90e28bc0a843b1a8485dafefcd4a05" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
I20260807 01:45:20.279167 25024 raft_consensus.cc:3055] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:45:20.279598 25024 raft_consensus.cc:740] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07a5248f4bf84ded832b7e261c3768b1, State: Running, Role: LEADER
I20260807 01:45:20.280602 25024 consensus_queue.cc:260] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:20.282294 24792 raft_consensus.cc:2468] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f22c8ad8eba94330950285be7e2a09a8 in term 2.
I20260807 01:45:20.283108 25024 raft_consensus.cc:3060] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:20.284314 24864 leader_election.cc:304] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9cb56177a511438f9f4a8c9ad5d64852, f22c8ad8eba94330950285be7e2a09a8; no voters:
I20260807 01:45:20.285418 25214 raft_consensus.cc:2804] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:45:20.286060 25214 raft_consensus.cc:697] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 2 LEADER]: Becoming Leader. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Running, Role: LEADER
I20260807 01:45:20.287319 25214 consensus_queue.cc:237] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:20.294108 25024 raft_consensus.cc:2468] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f22c8ad8eba94330950285be7e2a09a8 in term 2.
I20260807 01:45:20.299280 25216 consensus_queue.cc:1048] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:45:20.298959 24638 catalog_manager.cc:5697] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 reported cstate change: term changed from 1 to 2, leader changed from 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3) to f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2). New cstate: current_term: 2 leader_uuid: "f22c8ad8eba94330950285be7e2a09a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:20.319813 25198 raft_consensus.cc:993] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8: : Instructing follower 9cb56177a511438f9f4a8c9ad5d64852 to start an election
I20260807 01:45:20.320454 25198 raft_consensus.cc:1081] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 1 LEADER]: Signalling peer 9cb56177a511438f9f4a8c9ad5d64852 to start an election
I20260807 01:45:20.324676 24800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2791c7f6f9c647df909f735bd72c5a63"
dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
from {username='slave'} at 127.0.0.1:38844
I20260807 01:45:20.327973 24800 raft_consensus.cc:493] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:45:20.328516 25251 consensus_queue.cc:1048] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:45:20.331385 24800 raft_consensus.cc:3060] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:20.345554 24800 raft_consensus.cc:515] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:20.349012 24800 leader_election.cc:290] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 2 election: Requested vote from peers f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:20.350301 24912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2791c7f6f9c647df909f735bd72c5a63" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
I20260807 01:45:20.350965 25024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2791c7f6f9c647df909f735bd72c5a63" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
I20260807 01:45:20.351311 24912 raft_consensus.cc:3055] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:45:20.351846 25024 raft_consensus.cc:3060] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:20.352031 24912 raft_consensus.cc:740] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Running, Role: LEADER
I20260807 01:45:20.353228 24912 consensus_queue.cc:260] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:20.355155 24912 raft_consensus.cc:3060] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:20.363011 25024 raft_consensus.cc:2468] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb56177a511438f9f4a8c9ad5d64852 in term 2.
I20260807 01:45:20.364580 24738 leader_election.cc:304] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1, 9cb56177a511438f9f4a8c9ad5d64852; no voters:
I20260807 01:45:20.364974 24912 raft_consensus.cc:2468] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb56177a511438f9f4a8c9ad5d64852 in term 2.
I20260807 01:45:20.365854 25194 raft_consensus.cc:2804] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:45:20.366568 25194 raft_consensus.cc:697] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 LEADER]: Becoming Leader. State: Replica: 9cb56177a511438f9f4a8c9ad5d64852, State: Running, Role: LEADER
I20260807 01:45:20.367774 25194 consensus_queue.cc:237] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:20.381127 24636 catalog_manager.cc:5697] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 reported cstate change: term changed from 1 to 2, leader changed from f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2) to 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "9cb56177a511438f9f4a8c9ad5d64852" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:20.767143 24800 raft_consensus.cc:1275] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Refusing update from remote peer f22c8ad8eba94330950285be7e2a09a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:20.768762 25214 consensus_queue.cc:1048] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:45:20.779760 25024 raft_consensus.cc:1275] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 2 FOLLOWER]: Refusing update from remote peer f22c8ad8eba94330950285be7e2a09a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:20.781261 25198 consensus_queue.cc:1048] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:45:20.788359 24912 raft_consensus.cc:1275] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 2 FOLLOWER]: Refusing update from remote peer 9cb56177a511438f9f4a8c9ad5d64852: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:20.790333 25194 consensus_queue.cc:1048] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:45:20.801966 25024 raft_consensus.cc:1275] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 2 FOLLOWER]: Refusing update from remote peer 9cb56177a511438f9f4a8c9ad5d64852: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:20.803396 25257 consensus_queue.cc:1048] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:45:23.241323 24592 auto_leader_rebalancer-test.cc:402] The leader distribution is
I20260807 01:45:23.243330 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:23.287220 24792 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2791c7f6f9c647df909f735bd72c5a63"
dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
mode: GRACEFUL
new_leader_uuid: "07a5248f4bf84ded832b7e261c3768b1"
from {username='slave'} at 127.0.0.1:32984
I20260807 01:45:23.288287 24792 raft_consensus.cc:606] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 LEADER]: Received request to transfer leadership to TS 07a5248f4bf84ded832b7e261c3768b1
I20260807 01:45:23.311308 24912 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc90e28bc0a843b1a8485dafefcd4a05"
dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
mode: GRACEFUL
new_leader_uuid: "07a5248f4bf84ded832b7e261c3768b1"
from {username='slave'} at 127.0.0.1:41416
I20260807 01:45:23.312199 24912 raft_consensus.cc:606] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 2 LEADER]: Received request to transfer leadership to TS 07a5248f4bf84ded832b7e261c3768b1
I20260807 01:45:23.315158 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260807 01:45:23.316120 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:23.522003 25271 raft_consensus.cc:993] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8: : Instructing follower 07a5248f4bf84ded832b7e261c3768b1 to start an election
I20260807 01:45:23.523403 25268 raft_consensus.cc:1081] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 2 LEADER]: Signalling peer 07a5248f4bf84ded832b7e261c3768b1 to start an election
I20260807 01:45:23.527285 25024 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dc90e28bc0a843b1a8485dafefcd4a05"
dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
from {username='slave'} at 127.0.0.1:55840
I20260807 01:45:23.528755 25024 raft_consensus.cc:493] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:45:23.529793 25024 raft_consensus.cc:3060] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:45:23.541939 25024 raft_consensus.cc:515] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:23.546753 25024 leader_election.cc:290] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 3 election: Requested vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449)
I20260807 01:45:23.549293 24800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc90e28bc0a843b1a8485dafefcd4a05" candidate_uuid: "07a5248f4bf84ded832b7e261c3768b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
I20260807 01:45:23.550820 24800 raft_consensus.cc:3060] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:45:23.551403 24912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc90e28bc0a843b1a8485dafefcd4a05" candidate_uuid: "07a5248f4bf84ded832b7e261c3768b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
I20260807 01:45:23.552310 24912 raft_consensus.cc:3055] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:45:23.552738 24912 raft_consensus.cc:740] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Running, Role: LEADER
I20260807 01:45:23.554093 24912 consensus_queue.cc:260] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:23.556056 24912 raft_consensus.cc:3060] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:45:23.560350 24800 raft_consensus.cc:2468] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a5248f4bf84ded832b7e261c3768b1 in term 3.
I20260807 01:45:23.563212 24973 leader_election.cc:304] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1, 9cb56177a511438f9f4a8c9ad5d64852; no voters:
I20260807 01:45:23.565194 25275 raft_consensus.cc:2804] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:45:23.567624 24912 raft_consensus.cc:2468] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a5248f4bf84ded832b7e261c3768b1 in term 3.
I20260807 01:45:23.568128 25275 raft_consensus.cc:697] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 3 LEADER]: Becoming Leader. State: Replica: 07a5248f4bf84ded832b7e261c3768b1, State: Running, Role: LEADER
I20260807 01:45:23.570261 25275 consensus_queue.cc:237] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:23.585654 24639 catalog_manager.cc:5697] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 reported cstate change: term changed from 2 to 3, leader changed from f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2) to 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3). New cstate: current_term: 3 leader_uuid: "07a5248f4bf84ded832b7e261c3768b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } health_report { overall_health: HEALTHY } } }
I20260807 01:45:23.769441 25265 raft_consensus.cc:993] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852: : Instructing follower 07a5248f4bf84ded832b7e261c3768b1 to start an election
I20260807 01:45:23.770006 25265 raft_consensus.cc:1081] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 LEADER]: Signalling peer 07a5248f4bf84ded832b7e261c3768b1 to start an election
I20260807 01:45:23.772166 25024 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2791c7f6f9c647df909f735bd72c5a63"
dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
from {username='slave'} at 127.0.0.1:55834
I20260807 01:45:23.772861 25024 raft_consensus.cc:493] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:45:23.773228 25024 raft_consensus.cc:3060] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:45:23.780584 25024 raft_consensus.cc:515] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:23.782790 25024 leader_election.cc:290] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 3 election: Requested vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449)
I20260807 01:45:23.784327 24800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2791c7f6f9c647df909f735bd72c5a63" candidate_uuid: "07a5248f4bf84ded832b7e261c3768b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
I20260807 01:45:23.784793 24912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2791c7f6f9c647df909f735bd72c5a63" candidate_uuid: "07a5248f4bf84ded832b7e261c3768b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
I20260807 01:45:23.785059 24800 raft_consensus.cc:3055] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:45:23.785661 24800 raft_consensus.cc:740] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9cb56177a511438f9f4a8c9ad5d64852, State: Running, Role: LEADER
I20260807 01:45:23.785676 24912 raft_consensus.cc:3060] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:45:23.786890 24800 consensus_queue.cc:260] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:23.788753 24800 raft_consensus.cc:3060] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:45:23.793834 24912 raft_consensus.cc:2468] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a5248f4bf84ded832b7e261c3768b1 in term 3.
I20260807 01:45:23.795620 24974 leader_election.cc:304] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1, f22c8ad8eba94330950285be7e2a09a8; no voters:
I20260807 01:45:23.796598 25275 raft_consensus.cc:2804] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:45:23.796833 24800 raft_consensus.cc:2468] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a5248f4bf84ded832b7e261c3768b1 in term 3.
I20260807 01:45:23.797365 25275 raft_consensus.cc:697] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 3 LEADER]: Becoming Leader. State: Replica: 07a5248f4bf84ded832b7e261c3768b1, State: Running, Role: LEADER
I20260807 01:45:23.798601 25275 consensus_queue.cc:237] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:23.809528 24639 catalog_manager.cc:5697] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 reported cstate change: term changed from 2 to 3, leader changed from 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1) to 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3). New cstate: current_term: 3 leader_uuid: "07a5248f4bf84ded832b7e261c3768b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } health_report { overall_health: HEALTHY } } }
I20260807 01:45:23.980602 24800 raft_consensus.cc:1275] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852 [term 3 FOLLOWER]: Refusing update from remote peer 07a5248f4bf84ded832b7e261c3768b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:45:23.983037 25275 consensus_queue.cc:1048] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:45:24.020243 24912 raft_consensus.cc:1275] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 3 FOLLOWER]: Refusing update from remote peer 07a5248f4bf84ded832b7e261c3768b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:45:24.024260 25298 consensus_queue.cc:1048] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:45:24.302291 24800 raft_consensus.cc:1275] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 3 FOLLOWER]: Refusing update from remote peer 07a5248f4bf84ded832b7e261c3768b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:45:24.305332 25298 consensus_queue.cc:1048] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:45:24.318980 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:24.335482 24912 raft_consensus.cc:1275] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 3 FOLLOWER]: Refusing update from remote peer 07a5248f4bf84ded832b7e261c3768b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:45:24.338619 25298 consensus_queue.cc:1048] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:45:24.347682 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:45:24.349507 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:25.351616 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:25.361337 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:45:25.362324 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:26.363557 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:26.368779 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:45:26.369329 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:27.370379 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:27.375336 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:45:27.375844 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:28.376971 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:28.382671 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:45:28.383201 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:29.384157 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:29.390507 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:45:29.391178 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:30.392264 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:30.397509 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:45:30.397984 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:31.399075 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:31.404062 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:45:31.404557 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:32.405781 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:32.411598 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:45:32.412124 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:33.450184 24592 auto_leader_rebalancer-test.cc:421] The leader distribution is
I20260807 01:45:33.467085 24800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f0fd88947c65482b8e9d6b23e96a4db2"
dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
mode: GRACEFUL
new_leader_uuid: "f22c8ad8eba94330950285be7e2a09a8"
from {username='slave'} at 127.0.0.1:40158
I20260807 01:45:33.467777 24800 raft_consensus.cc:606] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 LEADER]: Received request to transfer leadership to TS f22c8ad8eba94330950285be7e2a09a8
I20260807 01:45:33.474718 25022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2791c7f6f9c647df909f735bd72c5a63"
dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
mode: GRACEFUL
new_leader_uuid: "f22c8ad8eba94330950285be7e2a09a8"
from {username='slave'} at 127.0.0.1:55850
I20260807 01:45:33.475610 25022 raft_consensus.cc:606] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 3 LEADER]: Received request to transfer leadership to TS f22c8ad8eba94330950285be7e2a09a8
I20260807 01:45:33.481909 24800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9052c9d021c5487da14be02e7852fb2a"
dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
mode: GRACEFUL
new_leader_uuid: "f22c8ad8eba94330950285be7e2a09a8"
from {username='slave'} at 127.0.0.1:40158
I20260807 01:45:33.482837 24800 raft_consensus.cc:606] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 LEADER]: Received request to transfer leadership to TS f22c8ad8eba94330950285be7e2a09a8
I20260807 01:45:33.488634 24800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "719f845bd45941dcaa2421158f4b0ebe"
dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
mode: GRACEFUL
new_leader_uuid: "f22c8ad8eba94330950285be7e2a09a8"
from {username='slave'} at 127.0.0.1:40158
I20260807 01:45:33.489326 24800 raft_consensus.cc:606] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 LEADER]: Received request to transfer leadership to TS f22c8ad8eba94330950285be7e2a09a8
I20260807 01:45:33.503827 25022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc90e28bc0a843b1a8485dafefcd4a05"
dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
mode: GRACEFUL
new_leader_uuid: "f22c8ad8eba94330950285be7e2a09a8"
from {username='slave'} at 127.0.0.1:55850
I20260807 01:45:33.504415 25022 raft_consensus.cc:606] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 3 LEADER]: Received request to transfer leadership to TS f22c8ad8eba94330950285be7e2a09a8
I20260807 01:45:33.580884 25312 raft_consensus.cc:993] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852: : Instructing follower f22c8ad8eba94330950285be7e2a09a8 to start an election
I20260807 01:45:33.581367 25312 raft_consensus.cc:1081] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 LEADER]: Signalling peer f22c8ad8eba94330950285be7e2a09a8 to start an election
I20260807 01:45:33.584012 24912 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9052c9d021c5487da14be02e7852fb2a"
dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
from {username='slave'} at 127.0.0.1:45650
I20260807 01:45:33.585172 24912 raft_consensus.cc:493] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:45:33.585997 24912 raft_consensus.cc:3060] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:33.596587 24912 raft_consensus.cc:515] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:33.599928 24912 leader_election.cc:290] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 2 election: Requested vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:33.601372 24800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9052c9d021c5487da14be02e7852fb2a" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
I20260807 01:45:33.601733 25022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9052c9d021c5487da14be02e7852fb2a" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
I20260807 01:45:33.602290 24800 raft_consensus.cc:3055] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:45:33.602415 25022 raft_consensus.cc:3060] T 9052c9d021c5487da14be02e7852fb2a P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:33.602560 24800 raft_consensus.cc:740] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9cb56177a511438f9f4a8c9ad5d64852, State: Running, Role: LEADER
I20260807 01:45:33.603557 24800 consensus_queue.cc:260] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:33.604722 24800 raft_consensus.cc:3060] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:33.612424 24800 raft_consensus.cc:2468] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f22c8ad8eba94330950285be7e2a09a8 in term 2.
I20260807 01:45:33.612560 25022 raft_consensus.cc:2468] T 9052c9d021c5487da14be02e7852fb2a P 07a5248f4bf84ded832b7e261c3768b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f22c8ad8eba94330950285be7e2a09a8 in term 2.
I20260807 01:45:33.614179 24864 leader_election.cc:304] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9cb56177a511438f9f4a8c9ad5d64852, f22c8ad8eba94330950285be7e2a09a8; no voters:
I20260807 01:45:33.615617 25271 raft_consensus.cc:2804] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:45:33.616825 25271 raft_consensus.cc:697] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [term 2 LEADER]: Becoming Leader. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Running, Role: LEADER
I20260807 01:45:33.617934 25271 consensus_queue.cc:237] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:33.631697 24639 catalog_manager.cc:5697] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 reported cstate change: term changed from 1 to 2, leader changed from 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1) to f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2). New cstate: current_term: 2 leader_uuid: "f22c8ad8eba94330950285be7e2a09a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:33.668949 25310 raft_consensus.cc:993] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852: : Instructing follower f22c8ad8eba94330950285be7e2a09a8 to start an election
I20260807 01:45:33.669772 25310 raft_consensus.cc:1081] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 LEADER]: Signalling peer f22c8ad8eba94330950285be7e2a09a8 to start an election
I20260807 01:45:33.672158 24912 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f0fd88947c65482b8e9d6b23e96a4db2"
dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
from {username='slave'} at 127.0.0.1:45650
I20260807 01:45:33.672627 24912 raft_consensus.cc:493] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:45:33.672879 24912 raft_consensus.cc:3060] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:33.677541 24912 raft_consensus.cc:515] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:33.679095 24912 leader_election.cc:290] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 2 election: Requested vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:33.680541 24800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0fd88947c65482b8e9d6b23e96a4db2" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
I20260807 01:45:33.681362 24800 raft_consensus.cc:3055] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:45:33.680540 25022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0fd88947c65482b8e9d6b23e96a4db2" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
I20260807 01:45:33.681728 24800 raft_consensus.cc:740] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9cb56177a511438f9f4a8c9ad5d64852, State: Running, Role: LEADER
I20260807 01:45:33.682036 25022 raft_consensus.cc:3060] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:33.682879 24800 consensus_queue.cc:260] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:33.684765 24800 raft_consensus.cc:3060] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:33.694276 24800 raft_consensus.cc:2468] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f22c8ad8eba94330950285be7e2a09a8 in term 2.
I20260807 01:45:33.694475 25022 raft_consensus.cc:2468] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f22c8ad8eba94330950285be7e2a09a8 in term 2.
I20260807 01:45:33.695672 24864 leader_election.cc:304] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9cb56177a511438f9f4a8c9ad5d64852, f22c8ad8eba94330950285be7e2a09a8; no voters:
I20260807 01:45:33.696542 25271 raft_consensus.cc:2804] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:45:33.697021 25271 raft_consensus.cc:697] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [term 2 LEADER]: Becoming Leader. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Running, Role: LEADER
I20260807 01:45:33.698007 25271 consensus_queue.cc:237] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:33.708424 24639 catalog_manager.cc:5697] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 reported cstate change: term changed from 1 to 2, leader changed from 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1) to f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2). New cstate: current_term: 2 leader_uuid: "f22c8ad8eba94330950285be7e2a09a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:33.802475 25312 raft_consensus.cc:993] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852: : Instructing follower f22c8ad8eba94330950285be7e2a09a8 to start an election
I20260807 01:45:33.803133 25310 raft_consensus.cc:1081] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 LEADER]: Signalling peer f22c8ad8eba94330950285be7e2a09a8 to start an election
I20260807 01:45:33.804920 24912 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "719f845bd45941dcaa2421158f4b0ebe"
dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
from {username='slave'} at 127.0.0.1:45650
I20260807 01:45:33.805366 24912 raft_consensus.cc:493] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:45:33.805634 24912 raft_consensus.cc:3060] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:33.809940 24912 raft_consensus.cc:515] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:33.811571 24912 leader_election.cc:290] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 2 election: Requested vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:33.812726 24800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719f845bd45941dcaa2421158f4b0ebe" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
I20260807 01:45:33.812959 25022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719f845bd45941dcaa2421158f4b0ebe" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
I20260807 01:45:33.813349 24800 raft_consensus.cc:3055] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:45:33.813522 25022 raft_consensus.cc:3060] T 719f845bd45941dcaa2421158f4b0ebe P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:33.813690 24800 raft_consensus.cc:740] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9cb56177a511438f9f4a8c9ad5d64852, State: Running, Role: LEADER
I20260807 01:45:33.814515 24800 consensus_queue.cc:260] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:33.815935 24800 raft_consensus.cc:3060] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:33.822144 25022 raft_consensus.cc:2468] T 719f845bd45941dcaa2421158f4b0ebe P 07a5248f4bf84ded832b7e261c3768b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f22c8ad8eba94330950285be7e2a09a8 in term 2.
I20260807 01:45:33.822875 24800 raft_consensus.cc:2468] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f22c8ad8eba94330950285be7e2a09a8 in term 2.
I20260807 01:45:33.823647 24861 leader_election.cc:304] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1, f22c8ad8eba94330950285be7e2a09a8; no voters:
I20260807 01:45:33.824555 25271 raft_consensus.cc:2804] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:45:33.824980 25271 raft_consensus.cc:697] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [term 2 LEADER]: Becoming Leader. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Running, Role: LEADER
I20260807 01:45:33.825836 25271 consensus_queue.cc:237] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:33.835538 24639 catalog_manager.cc:5697] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 reported cstate change: term changed from 1 to 2, leader changed from 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1) to f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2). New cstate: current_term: 2 leader_uuid: "f22c8ad8eba94330950285be7e2a09a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:33.940016 25309 raft_consensus.cc:993] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1: : Instructing follower f22c8ad8eba94330950285be7e2a09a8 to start an election
I20260807 01:45:33.940531 25309 raft_consensus.cc:1081] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 3 LEADER]: Signalling peer f22c8ad8eba94330950285be7e2a09a8 to start an election
I20260807 01:45:33.942695 24912 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dc90e28bc0a843b1a8485dafefcd4a05"
dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
from {username='slave'} at 127.0.0.1:45652
I20260807 01:45:33.943140 24912 raft_consensus.cc:493] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:45:33.943372 24912 raft_consensus.cc:3060] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:45:33.947548 24912 raft_consensus.cc:515] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:33.949026 24912 leader_election.cc:290] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 4 election: Requested vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:33.949977 24800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc90e28bc0a843b1a8485dafefcd4a05" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
I20260807 01:45:33.950276 25022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc90e28bc0a843b1a8485dafefcd4a05" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
I20260807 01:45:33.950618 24800 raft_consensus.cc:3060] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:45:33.950943 25022 raft_consensus.cc:3055] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:45:33.951288 25022 raft_consensus.cc:740] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 07a5248f4bf84ded832b7e261c3768b1, State: Running, Role: LEADER
I20260807 01:45:33.952025 25022 consensus_queue.cc:260] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:33.953306 25022 raft_consensus.cc:3060] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:45:33.955180 24800 raft_consensus.cc:2468] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f22c8ad8eba94330950285be7e2a09a8 in term 4.
I20260807 01:45:33.956300 24864 leader_election.cc:304] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9cb56177a511438f9f4a8c9ad5d64852, f22c8ad8eba94330950285be7e2a09a8; no voters:
I20260807 01:45:33.956998 25271 raft_consensus.cc:2804] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:45:33.957446 25271 raft_consensus.cc:697] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 4 LEADER]: Becoming Leader. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Running, Role: LEADER
I20260807 01:45:33.958294 25271 consensus_queue.cc:237] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:33.959587 25022 raft_consensus.cc:2468] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f22c8ad8eba94330950285be7e2a09a8 in term 4.
I20260807 01:45:33.967697 24639 catalog_manager.cc:5697] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 reported cstate change: term changed from 3 to 4, leader changed from 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3) to f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2). New cstate: current_term: 4 leader_uuid: "f22c8ad8eba94330950285be7e2a09a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:33.980582 25329 raft_consensus.cc:993] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1: : Instructing follower f22c8ad8eba94330950285be7e2a09a8 to start an election
I20260807 01:45:33.981118 25309 raft_consensus.cc:1081] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 3 LEADER]: Signalling peer f22c8ad8eba94330950285be7e2a09a8 to start an election
I20260807 01:45:33.982879 24912 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2791c7f6f9c647df909f735bd72c5a63"
dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
from {username='slave'} at 127.0.0.1:45652
I20260807 01:45:33.983345 24912 raft_consensus.cc:493] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:45:33.983606 24912 raft_consensus.cc:3060] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:45:33.988131 24912 raft_consensus.cc:515] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:33.990021 24912 leader_election.cc:290] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 4 election: Requested vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:33.990564 24800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2791c7f6f9c647df909f735bd72c5a63" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
I20260807 01:45:33.991191 24800 raft_consensus.cc:3060] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:45:33.991397 25022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2791c7f6f9c647df909f735bd72c5a63" candidate_uuid: "f22c8ad8eba94330950285be7e2a09a8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
I20260807 01:45:33.991926 25022 raft_consensus.cc:3055] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:45:33.992223 25022 raft_consensus.cc:740] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 07a5248f4bf84ded832b7e261c3768b1, State: Running, Role: LEADER
I20260807 01:45:33.993083 25022 consensus_queue.cc:260] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:33.994360 25022 raft_consensus.cc:3060] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:45:33.998063 24800 raft_consensus.cc:2468] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f22c8ad8eba94330950285be7e2a09a8 in term 4.
I20260807 01:45:33.999305 24864 leader_election.cc:304] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9cb56177a511438f9f4a8c9ad5d64852, f22c8ad8eba94330950285be7e2a09a8; no voters:
I20260807 01:45:33.999980 25271 raft_consensus.cc:2804] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:45:34.000177 25022 raft_consensus.cc:2468] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f22c8ad8eba94330950285be7e2a09a8 in term 4.
I20260807 01:45:34.000325 25271 raft_consensus.cc:697] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 4 LEADER]: Becoming Leader. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Running, Role: LEADER
I20260807 01:45:34.001155 25271 consensus_queue.cc:237] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:34.008574 24639 catalog_manager.cc:5697] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 reported cstate change: term changed from 3 to 4, leader changed from 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3) to f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2). New cstate: current_term: 4 leader_uuid: "f22c8ad8eba94330950285be7e2a09a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:34.080775 24800 raft_consensus.cc:1275] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Refusing update from remote peer f22c8ad8eba94330950285be7e2a09a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:34.082515 25271 consensus_queue.cc:1048] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:45:34.095046 25022 raft_consensus.cc:1275] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1 [term 2 FOLLOWER]: Refusing update from remote peer f22c8ad8eba94330950285be7e2a09a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:34.099517 25339 consensus_queue.cc:1048] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:45:34.130373 25022 raft_consensus.cc:1275] T 9052c9d021c5487da14be02e7852fb2a P 07a5248f4bf84ded832b7e261c3768b1 [term 2 FOLLOWER]: Refusing update from remote peer f22c8ad8eba94330950285be7e2a09a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:34.131963 25339 consensus_queue.cc:1048] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:45:34.139810 24800 raft_consensus.cc:1275] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Refusing update from remote peer f22c8ad8eba94330950285be7e2a09a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:34.141314 25339 consensus_queue.cc:1048] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:45:34.260495 25022 raft_consensus.cc:1275] T 719f845bd45941dcaa2421158f4b0ebe P 07a5248f4bf84ded832b7e261c3768b1 [term 2 FOLLOWER]: Refusing update from remote peer f22c8ad8eba94330950285be7e2a09a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:34.261986 25339 consensus_queue.cc:1048] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:45:34.269733 24800 raft_consensus.cc:1275] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Refusing update from remote peer f22c8ad8eba94330950285be7e2a09a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:34.271764 25316 consensus_queue.cc:1048] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:45:34.406500 24800 raft_consensus.cc:1275] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852 [term 4 FOLLOWER]: Refusing update from remote peer f22c8ad8eba94330950285be7e2a09a8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:45:34.407837 25271 consensus_queue.cc:1048] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:45:34.418951 25022 raft_consensus.cc:1275] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 4 FOLLOWER]: Refusing update from remote peer f22c8ad8eba94330950285be7e2a09a8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:45:34.420430 25316 consensus_queue.cc:1048] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:45:34.498057 25022 raft_consensus.cc:1275] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 4 FOLLOWER]: Refusing update from remote peer f22c8ad8eba94330950285be7e2a09a8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:45:34.499737 25316 consensus_queue.cc:1048] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:45:34.508592 24800 raft_consensus.cc:1275] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 4 FOLLOWER]: Refusing update from remote peer f22c8ad8eba94330950285be7e2a09a8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:45:34.510682 25271 consensus_queue.cc:1048] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:45:36.542914 24592 auto_leader_rebalancer-test.cc:436] The leader distribution is
I20260807 01:45:36.544070 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:36.551561 24912 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "088984f6ac834d64b99408685df54428"
dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
mode: GRACEFUL
new_leader_uuid: "07a5248f4bf84ded832b7e261c3768b1"
from {username='slave'} at 127.0.0.1:41416
I20260807 01:45:36.552191 24912 raft_consensus.cc:606] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [term 1 LEADER]: Received request to transfer leadership to TS 07a5248f4bf84ded832b7e261c3768b1
I20260807 01:45:36.555107 24912 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "719f845bd45941dcaa2421158f4b0ebe"
dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
mode: GRACEFUL
new_leader_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
from {username='slave'} at 127.0.0.1:41416
I20260807 01:45:36.555557 24912 raft_consensus.cc:606] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [term 2 LEADER]: Received request to transfer leadership to TS 9cb56177a511438f9f4a8c9ad5d64852
I20260807 01:45:36.557907 24912 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9052c9d021c5487da14be02e7852fb2a"
dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
mode: GRACEFUL
new_leader_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
from {username='slave'} at 127.0.0.1:41416
I20260807 01:45:36.558409 24912 raft_consensus.cc:606] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [term 2 LEADER]: Received request to transfer leadership to TS 9cb56177a511438f9f4a8c9ad5d64852
I20260807 01:45:36.561182 24912 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b606cc1d46c3462f9759d92957b384f0"
dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
mode: GRACEFUL
new_leader_uuid: "07a5248f4bf84ded832b7e261c3768b1"
from {username='slave'} at 127.0.0.1:41416
I20260807 01:45:36.561833 24912 raft_consensus.cc:606] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [term 1 LEADER]: Received request to transfer leadership to TS 07a5248f4bf84ded832b7e261c3768b1
I20260807 01:45:36.565153 24912 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f0fd88947c65482b8e9d6b23e96a4db2"
dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
mode: GRACEFUL
new_leader_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
from {username='slave'} at 127.0.0.1:41416
I20260807 01:45:36.565726 24912 raft_consensus.cc:606] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [term 2 LEADER]: Received request to transfer leadership to TS 9cb56177a511438f9f4a8c9ad5d64852
I20260807 01:45:36.567270 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260807 01:45:36.567902 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:36.577000 25316 raft_consensus.cc:993] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8: : Instructing follower 9cb56177a511438f9f4a8c9ad5d64852 to start an election
I20260807 01:45:36.577523 25316 raft_consensus.cc:1081] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [term 2 LEADER]: Signalling peer 9cb56177a511438f9f4a8c9ad5d64852 to start an election
I20260807 01:45:36.579423 24800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9052c9d021c5487da14be02e7852fb2a"
dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
from {username='slave'} at 127.0.0.1:38844
I20260807 01:45:36.579905 24800 raft_consensus.cc:493] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:45:36.580164 24800 raft_consensus.cc:3060] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:45:36.587054 24800 raft_consensus.cc:515] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:36.589578 24800 leader_election.cc:290] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 3 election: Requested vote from peers f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:36.590647 24912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9052c9d021c5487da14be02e7852fb2a" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
I20260807 01:45:36.590720 25022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9052c9d021c5487da14be02e7852fb2a" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
I20260807 01:45:36.591323 24912 raft_consensus.cc:3055] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:45:36.591338 25022 raft_consensus.cc:3060] T 9052c9d021c5487da14be02e7852fb2a P 07a5248f4bf84ded832b7e261c3768b1 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:45:36.591673 24912 raft_consensus.cc:740] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Running, Role: LEADER
I20260807 01:45:36.592280 24912 consensus_queue.cc:260] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:36.593137 24912 raft_consensus.cc:3060] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:45:36.596479 25022 raft_consensus.cc:2468] T 9052c9d021c5487da14be02e7852fb2a P 07a5248f4bf84ded832b7e261c3768b1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb56177a511438f9f4a8c9ad5d64852 in term 3.
I20260807 01:45:36.597623 24912 raft_consensus.cc:2468] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb56177a511438f9f4a8c9ad5d64852 in term 3.
I20260807 01:45:36.597856 24738 leader_election.cc:304] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1, 9cb56177a511438f9f4a8c9ad5d64852; no voters:
I20260807 01:45:36.599586 25364 raft_consensus.cc:2804] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:45:36.600600 25364 raft_consensus.cc:697] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 3 LEADER]: Becoming Leader. State: Replica: 9cb56177a511438f9f4a8c9ad5d64852, State: Running, Role: LEADER
I20260807 01:45:36.601400 25364 consensus_queue.cc:237] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:36.608825 24639 catalog_manager.cc:5697] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 reported cstate change: term changed from 2 to 3, leader changed from f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2) to 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1). New cstate: current_term: 3 leader_uuid: "9cb56177a511438f9f4a8c9ad5d64852" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:36.697304 25316 raft_consensus.cc:993] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8: : Instructing follower 07a5248f4bf84ded832b7e261c3768b1 to start an election
I20260807 01:45:36.697964 25361 raft_consensus.cc:1081] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [term 1 LEADER]: Signalling peer 07a5248f4bf84ded832b7e261c3768b1 to start an election
I20260807 01:45:36.699918 25022 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b606cc1d46c3462f9759d92957b384f0"
dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
from {username='slave'} at 127.0.0.1:55840
I20260807 01:45:36.700520 25022 raft_consensus.cc:493] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:45:36.700850 25022 raft_consensus.cc:3060] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:36.705188 25022 raft_consensus.cc:515] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:36.707126 25022 leader_election.cc:290] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 2 election: Requested vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449)
I20260807 01:45:36.708169 24800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b606cc1d46c3462f9759d92957b384f0" candidate_uuid: "07a5248f4bf84ded832b7e261c3768b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
I20260807 01:45:36.708310 24912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b606cc1d46c3462f9759d92957b384f0" candidate_uuid: "07a5248f4bf84ded832b7e261c3768b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
I20260807 01:45:36.708788 24800 raft_consensus.cc:3060] T b606cc1d46c3462f9759d92957b384f0 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:36.708838 24912 raft_consensus.cc:3055] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:45:36.709266 24912 raft_consensus.cc:740] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Running, Role: LEADER
I20260807 01:45:36.710032 24912 consensus_queue.cc:260] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:36.711221 24912 raft_consensus.cc:3060] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:36.715483 24800 raft_consensus.cc:2468] T b606cc1d46c3462f9759d92957b384f0 P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a5248f4bf84ded832b7e261c3768b1 in term 2.
I20260807 01:45:36.716921 24973 leader_election.cc:304] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1, 9cb56177a511438f9f4a8c9ad5d64852; no voters:
I20260807 01:45:36.717106 24912 raft_consensus.cc:2468] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a5248f4bf84ded832b7e261c3768b1 in term 2.
I20260807 01:45:36.717621 25358 raft_consensus.cc:2804] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:45:36.718852 25358 raft_consensus.cc:697] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1 [term 2 LEADER]: Becoming Leader. State: Replica: 07a5248f4bf84ded832b7e261c3768b1, State: Running, Role: LEADER
I20260807 01:45:36.719584 25358 consensus_queue.cc:237] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:36.726941 24639 catalog_manager.cc:5697] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1 reported cstate change: term changed from 1 to 2, leader changed from f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2) to 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3). New cstate: current_term: 2 leader_uuid: "07a5248f4bf84ded832b7e261c3768b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } health_report { overall_health: HEALTHY } } }
I20260807 01:45:36.830952 25361 raft_consensus.cc:993] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8: : Instructing follower 9cb56177a511438f9f4a8c9ad5d64852 to start an election
I20260807 01:45:36.831581 25381 raft_consensus.cc:1081] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [term 2 LEADER]: Signalling peer 9cb56177a511438f9f4a8c9ad5d64852 to start an election
I20260807 01:45:36.833844 24800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f0fd88947c65482b8e9d6b23e96a4db2"
dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
from {username='slave'} at 127.0.0.1:38844
I20260807 01:45:36.834568 24800 raft_consensus.cc:493] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:45:36.834944 24800 raft_consensus.cc:3060] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:45:36.839903 24800 raft_consensus.cc:515] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:36.841415 24800 leader_election.cc:290] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 3 election: Requested vote from peers f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:36.842473 24912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0fd88947c65482b8e9d6b23e96a4db2" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
I20260807 01:45:36.842617 25022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0fd88947c65482b8e9d6b23e96a4db2" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
I20260807 01:45:36.842998 24912 raft_consensus.cc:3055] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:45:36.843179 25022 raft_consensus.cc:3060] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:45:36.843321 24912 raft_consensus.cc:740] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Running, Role: LEADER
I20260807 01:45:36.843993 24912 consensus_queue.cc:260] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:36.844964 24912 raft_consensus.cc:3060] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:45:36.848248 25022 raft_consensus.cc:2468] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb56177a511438f9f4a8c9ad5d64852 in term 3.
I20260807 01:45:36.849463 24738 leader_election.cc:304] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1, 9cb56177a511438f9f4a8c9ad5d64852; no voters:
I20260807 01:45:36.850111 24912 raft_consensus.cc:2468] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb56177a511438f9f4a8c9ad5d64852 in term 3.
I20260807 01:45:36.850239 25364 raft_consensus.cc:2804] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:45:36.850811 25364 raft_consensus.cc:697] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 3 LEADER]: Becoming Leader. State: Replica: 9cb56177a511438f9f4a8c9ad5d64852, State: Running, Role: LEADER
I20260807 01:45:36.851619 25364 consensus_queue.cc:237] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:36.858489 24639 catalog_manager.cc:5697] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 reported cstate change: term changed from 2 to 3, leader changed from f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2) to 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1). New cstate: current_term: 3 leader_uuid: "9cb56177a511438f9f4a8c9ad5d64852" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:37.015611 25361 raft_consensus.cc:993] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8: : Instructing follower 07a5248f4bf84ded832b7e261c3768b1 to start an election
I20260807 01:45:37.016052 25361 raft_consensus.cc:1081] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [term 1 LEADER]: Signalling peer 07a5248f4bf84ded832b7e261c3768b1 to start an election
I20260807 01:45:37.017463 25022 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "088984f6ac834d64b99408685df54428"
dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
from {username='slave'} at 127.0.0.1:55840
I20260807 01:45:37.017920 25022 raft_consensus.cc:493] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:45:37.018224 25022 raft_consensus.cc:3060] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:37.022492 25022 raft_consensus.cc:515] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:37.024078 25022 leader_election.cc:290] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 2 election: Requested vote from peers 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099), f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449)
I20260807 01:45:37.024874 24800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "088984f6ac834d64b99408685df54428" candidate_uuid: "07a5248f4bf84ded832b7e261c3768b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
I20260807 01:45:37.025098 24912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "088984f6ac834d64b99408685df54428" candidate_uuid: "07a5248f4bf84ded832b7e261c3768b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
I20260807 01:45:37.025477 24800 raft_consensus.cc:3060] T 088984f6ac834d64b99408685df54428 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:37.025898 24912 raft_consensus.cc:3055] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:45:37.026264 24912 raft_consensus.cc:740] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Running, Role: LEADER
I20260807 01:45:37.027020 24912 consensus_queue.cc:260] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:37.028039 24912 raft_consensus.cc:3060] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:37.030068 24800 raft_consensus.cc:2468] T 088984f6ac834d64b99408685df54428 P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a5248f4bf84ded832b7e261c3768b1 in term 2.
I20260807 01:45:37.031208 24973 leader_election.cc:304] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1, 9cb56177a511438f9f4a8c9ad5d64852; no voters:
I20260807 01:45:37.031936 25358 raft_consensus.cc:2804] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:45:37.032439 25358 raft_consensus.cc:697] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 [term 2 LEADER]: Becoming Leader. State: Replica: 07a5248f4bf84ded832b7e261c3768b1, State: Running, Role: LEADER
I20260807 01:45:37.033305 24912 raft_consensus.cc:2468] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a5248f4bf84ded832b7e261c3768b1 in term 2.
I20260807 01:45:37.033274 25358 consensus_queue.cc:237] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:37.040047 25022 raft_consensus.cc:1275] T 9052c9d021c5487da14be02e7852fb2a P 07a5248f4bf84ded832b7e261c3768b1 [term 3 FOLLOWER]: Refusing update from remote peer 9cb56177a511438f9f4a8c9ad5d64852: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:45:37.041399 25364 consensus_queue.cc:1048] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:45:37.043499 24639 catalog_manager.cc:5697] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 reported cstate change: term changed from 1 to 2, leader changed from f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2) to 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3). New cstate: current_term: 2 leader_uuid: "07a5248f4bf84ded832b7e261c3768b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } health_report { overall_health: HEALTHY } } }
I20260807 01:45:37.051433 24912 raft_consensus.cc:1275] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [term 3 FOLLOWER]: Refusing update from remote peer 9cb56177a511438f9f4a8c9ad5d64852: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:45:37.053010 25368 consensus_queue.cc:1048] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:45:37.112030 25361 raft_consensus.cc:993] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8: : Instructing follower 9cb56177a511438f9f4a8c9ad5d64852 to start an election
I20260807 01:45:37.112502 25381 raft_consensus.cc:1081] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [term 2 LEADER]: Signalling peer 9cb56177a511438f9f4a8c9ad5d64852 to start an election
I20260807 01:45:37.113940 24800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "719f845bd45941dcaa2421158f4b0ebe"
dest_uuid: "9cb56177a511438f9f4a8c9ad5d64852"
from {username='slave'} at 127.0.0.1:38844
I20260807 01:45:37.114435 24800 raft_consensus.cc:493] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:45:37.114678 24800 raft_consensus.cc:3060] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:45:37.120537 24800 raft_consensus.cc:515] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:37.122143 24800 leader_election.cc:290] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 3 election: Requested vote from peers f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2:44449), 07a5248f4bf84ded832b7e261c3768b1 (127.24.4.3:38161)
I20260807 01:45:37.123067 24912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719f845bd45941dcaa2421158f4b0ebe" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f22c8ad8eba94330950285be7e2a09a8"
I20260807 01:45:37.123248 25022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719f845bd45941dcaa2421158f4b0ebe" candidate_uuid: "9cb56177a511438f9f4a8c9ad5d64852" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "07a5248f4bf84ded832b7e261c3768b1"
I20260807 01:45:37.123907 25022 raft_consensus.cc:3060] T 719f845bd45941dcaa2421158f4b0ebe P 07a5248f4bf84ded832b7e261c3768b1 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:45:37.123778 24912 raft_consensus.cc:3055] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:45:37.124325 24912 raft_consensus.cc:740] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f22c8ad8eba94330950285be7e2a09a8, State: Running, Role: LEADER
I20260807 01:45:37.124980 24912 consensus_queue.cc:260] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:37.125859 24912 raft_consensus.cc:3060] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:45:37.130127 25022 raft_consensus.cc:2468] T 719f845bd45941dcaa2421158f4b0ebe P 07a5248f4bf84ded832b7e261c3768b1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb56177a511438f9f4a8c9ad5d64852 in term 3.
I20260807 01:45:37.131428 24912 raft_consensus.cc:2468] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb56177a511438f9f4a8c9ad5d64852 in term 3.
I20260807 01:45:37.131559 24738 leader_election.cc:304] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a5248f4bf84ded832b7e261c3768b1, 9cb56177a511438f9f4a8c9ad5d64852; no voters:
I20260807 01:45:37.132405 25368 raft_consensus.cc:2804] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:45:37.132828 25368 raft_consensus.cc:697] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 3 LEADER]: Becoming Leader. State: Replica: 9cb56177a511438f9f4a8c9ad5d64852, State: Running, Role: LEADER
I20260807 01:45:37.133464 25368 consensus_queue.cc:237] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } }
I20260807 01:45:37.141000 24639 catalog_manager.cc:5697] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 reported cstate change: term changed from 2 to 3, leader changed from f22c8ad8eba94330950285be7e2a09a8 (127.24.4.2) to 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1). New cstate: current_term: 3 leader_uuid: "9cb56177a511438f9f4a8c9ad5d64852" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:37.257567 24912 raft_consensus.cc:1275] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [term 2 FOLLOWER]: Refusing update from remote peer 07a5248f4bf84ded832b7e261c3768b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:37.258916 25358 consensus_queue.cc:1048] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:45:37.262653 24800 raft_consensus.cc:1275] T b606cc1d46c3462f9759d92957b384f0 P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Refusing update from remote peer 07a5248f4bf84ded832b7e261c3768b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:37.264140 25358 consensus_queue.cc:1048] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:45:37.429626 25022 raft_consensus.cc:1275] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1 [term 3 FOLLOWER]: Refusing update from remote peer 9cb56177a511438f9f4a8c9ad5d64852: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:45:37.431071 25368 consensus_queue.cc:1048] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:45:37.439216 24912 raft_consensus.cc:1275] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [term 3 FOLLOWER]: Refusing update from remote peer 9cb56177a511438f9f4a8c9ad5d64852: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:45:37.440656 25368 consensus_queue.cc:1048] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:45:37.454659 24912 raft_consensus.cc:1275] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [term 2 FOLLOWER]: Refusing update from remote peer 07a5248f4bf84ded832b7e261c3768b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:37.456038 25378 consensus_queue.cc:1048] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:45:37.466780 24800 raft_consensus.cc:1275] T 088984f6ac834d64b99408685df54428 P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Refusing update from remote peer 07a5248f4bf84ded832b7e261c3768b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:37.469193 25395 consensus_queue.cc:1048] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb56177a511438f9f4a8c9ad5d64852" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:45:37.566341 25022 raft_consensus.cc:1275] T 719f845bd45941dcaa2421158f4b0ebe P 07a5248f4bf84ded832b7e261c3768b1 [term 3 FOLLOWER]: Refusing update from remote peer 9cb56177a511438f9f4a8c9ad5d64852: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:45:37.567728 25401 consensus_queue.cc:1048] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a5248f4bf84ded832b7e261c3768b1" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 38161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:45:37.569329 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:37.578188 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:45:37.578924 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:37.579133 24912 raft_consensus.cc:1275] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [term 3 FOLLOWER]: Refusing update from remote peer 9cb56177a511438f9f4a8c9ad5d64852: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:45:37.581378 25368 consensus_queue.cc:1048] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f22c8ad8eba94330950285be7e2a09a8" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:45:38.580622 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:38.586529 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:45:38.587085 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:39.588316 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:39.593567 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:45:39.594223 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:40.595408 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:40.601629 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:45:40.602273 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:41.603478 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:41.609097 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:45:41.609634 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:42.610785 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:42.615851 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:45:42.616366 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:43.617828 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:43.624763 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:45:43.625221 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:44.626617 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:44.631585 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:45:44.632131 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:45.633327 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:45.639065 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:45:45.639655 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:46.685168 24592 auto_leader_rebalancer-test.cc:448] The leader distribution is
I20260807 01:45:46.686142 24592 tablet_server.cc:179] TabletServer@127.24.4.1:0 shutting down...
I20260807 01:45:46.740263 24592 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260807 01:45:46.741787 24592 tablet_replica.cc:333] T 78946d0ad61f46a7a81f1592fa57a8d3 P 9cb56177a511438f9f4a8c9ad5d64852: stopping tablet replica
I20260807 01:45:46.742928 24592 raft_consensus.cc:2243] T 78946d0ad61f46a7a81f1592fa57a8d3 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 01:45:46.743850 24592 raft_consensus.cc:2272] T 78946d0ad61f46a7a81f1592fa57a8d3 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:46.747403 24592 tablet_replica.cc:333] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852: stopping tablet replica
I20260807 01:45:46.748418 24592 raft_consensus.cc:2243] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 01:45:46.749163 24592 raft_consensus.cc:2272] T dc90e28bc0a843b1a8485dafefcd4a05 P 9cb56177a511438f9f4a8c9ad5d64852 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:46.751490 24592 tablet_replica.cc:333] T d3c14e19345c4777a5f505485d89b588 P 9cb56177a511438f9f4a8c9ad5d64852: stopping tablet replica
I20260807 01:45:46.752115 24592 raft_consensus.cc:2243] T d3c14e19345c4777a5f505485d89b588 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 01:45:46.752630 24592 raft_consensus.cc:2272] T d3c14e19345c4777a5f505485d89b588 P 9cb56177a511438f9f4a8c9ad5d64852 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 01:45:46.753444 24973 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260807 01:45:46.754850 24592 tablet_replica.cc:333] T b606cc1d46c3462f9759d92957b384f0 P 9cb56177a511438f9f4a8c9ad5d64852: stopping tablet replica
I20260807 01:45:46.755515 24592 raft_consensus.cc:2243] T b606cc1d46c3462f9759d92957b384f0 P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:45:46.756049 24592 raft_consensus.cc:2272] T b606cc1d46c3462f9759d92957b384f0 P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260807 01:45:46.757552 24973 consensus_peers.cc:597] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 -> Peer 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099): Couldn't send request to peer 9cb56177a511438f9f4a8c9ad5d64852. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:45:46.758553 24592 tablet_replica.cc:333] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852: stopping tablet replica
I20260807 01:45:46.759272 24592 raft_consensus.cc:2243] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 3 LEADER]: Raft consensus shutting down.
I20260807 01:45:46.760452 24592 raft_consensus.cc:2272] T f0fd88947c65482b8e9d6b23e96a4db2 P 9cb56177a511438f9f4a8c9ad5d64852 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260807 01:45:46.763588 24864 consensus_peers.cc:597] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 -> Peer 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099): Couldn't send request to peer 9cb56177a511438f9f4a8c9ad5d64852. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:45:46.764238 24592 tablet_replica.cc:333] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852: stopping tablet replica
I20260807 01:45:46.765218 24592 raft_consensus.cc:2243] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 01:45:46.765884 24592 raft_consensus.cc:2272] T 2791c7f6f9c647df909f735bd72c5a63 P 9cb56177a511438f9f4a8c9ad5d64852 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:46.768579 24592 tablet_replica.cc:333] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852: stopping tablet replica
I20260807 01:45:46.769129 24592 raft_consensus.cc:2243] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 3 LEADER]: Raft consensus shutting down.
I20260807 01:45:46.770030 24592 raft_consensus.cc:2272] T 719f845bd45941dcaa2421158f4b0ebe P 9cb56177a511438f9f4a8c9ad5d64852 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:46.772459 24592 tablet_replica.cc:333] T 088984f6ac834d64b99408685df54428 P 9cb56177a511438f9f4a8c9ad5d64852: stopping tablet replica
I20260807 01:45:46.773063 24592 raft_consensus.cc:2243] T 088984f6ac834d64b99408685df54428 P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:45:46.773566 24592 raft_consensus.cc:2272] T 088984f6ac834d64b99408685df54428 P 9cb56177a511438f9f4a8c9ad5d64852 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:46.775519 24592 tablet_replica.cc:333] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852: stopping tablet replica
I20260807 01:45:46.776023 24592 raft_consensus.cc:2243] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 3 LEADER]: Raft consensus shutting down.
I20260807 01:45:46.776852 24592 raft_consensus.cc:2272] T 9052c9d021c5487da14be02e7852fb2a P 9cb56177a511438f9f4a8c9ad5d64852 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:46.976626 24592 tablet_server.cc:196] TabletServer@127.24.4.1:0 shutdown complete.
W20260807 01:45:46.978700 24973 consensus_peers.cc:597] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1 -> Peer 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099): Couldn't send request to peer 9cb56177a511438f9f4a8c9ad5d64852. Status: Network error: Client connection negotiation failed: client connection to 127.24.4.1:43099: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260807 01:45:46.979211 24973 consensus_peers.cc:597] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 -> Peer 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099): Couldn't send request to peer 9cb56177a511438f9f4a8c9ad5d64852. Status: Network error: Client connection negotiation failed: client connection to 127.24.4.1:43099: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260807 01:45:46.979565 24864 consensus_peers.cc:597] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 -> Peer 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099): Couldn't send request to peer 9cb56177a511438f9f4a8c9ad5d64852. Status: Network error: Client connection negotiation failed: client connection to 127.24.4.1:43099: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260807 01:45:46.979980 24864 consensus_peers.cc:597] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 -> Peer 9cb56177a511438f9f4a8c9ad5d64852 (127.24.4.1:43099): Couldn't send request to peer 9cb56177a511438f9f4a8c9ad5d64852. Status: Network error: Client connection negotiation failed: client connection to 127.24.4.1:43099: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 01:45:47.000707 24592 tablet_server.cc:179] TabletServer@127.24.4.2:0 shutting down...
I20260807 01:45:47.042732 24592 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260807 01:45:47.043509 24592 tablet_replica.cc:333] T 78946d0ad61f46a7a81f1592fa57a8d3 P f22c8ad8eba94330950285be7e2a09a8: stopping tablet replica
I20260807 01:45:47.044371 24592 raft_consensus.cc:2243] T 78946d0ad61f46a7a81f1592fa57a8d3 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 01:45:47.045013 24592 raft_consensus.cc:2272] T 78946d0ad61f46a7a81f1592fa57a8d3 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:47.048101 24592 tablet_replica.cc:333] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8: stopping tablet replica
I20260807 01:45:47.048681 24592 raft_consensus.cc:2243] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 4 LEADER]: Raft consensus shutting down.
I20260807 01:45:47.049539 24592 raft_consensus.cc:2272] T dc90e28bc0a843b1a8485dafefcd4a05 P f22c8ad8eba94330950285be7e2a09a8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:47.051542 24592 tablet_replica.cc:333] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8: stopping tablet replica
I20260807 01:45:47.052037 24592 raft_consensus.cc:2243] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 [term 1 LEADER]: Raft consensus shutting down.
I20260807 01:45:47.052860 24592 raft_consensus.cc:2272] T d3c14e19345c4777a5f505485d89b588 P f22c8ad8eba94330950285be7e2a09a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:47.054699 24592 tablet_replica.cc:333] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8: stopping tablet replica
I20260807 01:45:47.055217 24592 raft_consensus.cc:2243] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 01:45:47.055655 24592 raft_consensus.cc:2272] T f0fd88947c65482b8e9d6b23e96a4db2 P f22c8ad8eba94330950285be7e2a09a8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:47.057380 24592 tablet_replica.cc:333] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8: stopping tablet replica
I20260807 01:45:47.057868 24592 raft_consensus.cc:2243] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 4 LEADER]: Raft consensus shutting down.
I20260807 01:45:47.058717 24592 raft_consensus.cc:2272] T 2791c7f6f9c647df909f735bd72c5a63 P f22c8ad8eba94330950285be7e2a09a8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:47.060665 24592 tablet_replica.cc:333] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8: stopping tablet replica
I20260807 01:45:47.061152 24592 raft_consensus.cc:2243] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 01:45:47.061551 24592 raft_consensus.cc:2272] T 719f845bd45941dcaa2421158f4b0ebe P f22c8ad8eba94330950285be7e2a09a8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:47.063233 24592 tablet_replica.cc:333] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8: stopping tablet replica
I20260807 01:45:47.063720 24592 raft_consensus.cc:2243] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:45:47.064141 24592 raft_consensus.cc:2272] T 088984f6ac834d64b99408685df54428 P f22c8ad8eba94330950285be7e2a09a8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:47.065900 24592 tablet_replica.cc:333] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8: stopping tablet replica
I20260807 01:45:47.066428 24592 raft_consensus.cc:2243] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:45:47.066838 24592 raft_consensus.cc:2272] T b606cc1d46c3462f9759d92957b384f0 P f22c8ad8eba94330950285be7e2a09a8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:47.068468 24592 tablet_replica.cc:333] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8: stopping tablet replica
I20260807 01:45:47.068888 24592 raft_consensus.cc:2243] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 01:45:47.069378 24592 raft_consensus.cc:2272] T 9052c9d021c5487da14be02e7852fb2a P f22c8ad8eba94330950285be7e2a09a8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:47.109377 24592 tablet_server.cc:196] TabletServer@127.24.4.2:0 shutdown complete.
I20260807 01:45:47.139002 24592 tablet_server.cc:179] TabletServer@127.24.4.3:0 shutting down...
I20260807 01:45:47.173378 24592 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260807 01:45:47.174147 24592 tablet_replica.cc:333] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1: stopping tablet replica
I20260807 01:45:47.175062 24592 raft_consensus.cc:2243] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 01:45:47.176208 24592 raft_consensus.cc:2272] T 78946d0ad61f46a7a81f1592fa57a8d3 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:47.178336 24592 tablet_replica.cc:333] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1: stopping tablet replica
I20260807 01:45:47.178833 24592 raft_consensus.cc:2243] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 01:45:47.179232 24592 raft_consensus.cc:2272] T dc90e28bc0a843b1a8485dafefcd4a05 P 07a5248f4bf84ded832b7e261c3768b1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:47.180892 24592 tablet_replica.cc:333] T d3c14e19345c4777a5f505485d89b588 P 07a5248f4bf84ded832b7e261c3768b1: stopping tablet replica
I20260807 01:45:47.181365 24592 raft_consensus.cc:2243] T d3c14e19345c4777a5f505485d89b588 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 01:45:47.181728 24592 raft_consensus.cc:2272] T d3c14e19345c4777a5f505485d89b588 P 07a5248f4bf84ded832b7e261c3768b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:47.183401 24592 tablet_replica.cc:333] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1: stopping tablet replica
I20260807 01:45:47.183856 24592 raft_consensus.cc:2243] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 [term 2 LEADER]: Raft consensus shutting down.
I20260807 01:45:47.184635 24592 raft_consensus.cc:2272] T 088984f6ac834d64b99408685df54428 P 07a5248f4bf84ded832b7e261c3768b1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:47.186457 24592 tablet_replica.cc:333] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1: stopping tablet replica
I20260807 01:45:47.186926 24592 raft_consensus.cc:2243] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1 [term 2 LEADER]: Raft consensus shutting down.
I20260807 01:45:47.187668 24592 raft_consensus.cc:2272] T b606cc1d46c3462f9759d92957b384f0 P 07a5248f4bf84ded832b7e261c3768b1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:47.189311 24592 tablet_replica.cc:333] T 719f845bd45941dcaa2421158f4b0ebe P 07a5248f4bf84ded832b7e261c3768b1: stopping tablet replica
I20260807 01:45:47.189972 24592 raft_consensus.cc:2243] T 719f845bd45941dcaa2421158f4b0ebe P 07a5248f4bf84ded832b7e261c3768b1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 01:45:47.190455 24592 raft_consensus.cc:2272] T 719f845bd45941dcaa2421158f4b0ebe P 07a5248f4bf84ded832b7e261c3768b1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:47.192181 24592 tablet_replica.cc:333] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1: stopping tablet replica
I20260807 01:45:47.192646 24592 raft_consensus.cc:2243] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 01:45:47.193077 24592 raft_consensus.cc:2272] T 2791c7f6f9c647df909f735bd72c5a63 P 07a5248f4bf84ded832b7e261c3768b1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:47.194789 24592 tablet_replica.cc:333] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1: stopping tablet replica
I20260807 01:45:47.195235 24592 raft_consensus.cc:2243] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 01:45:47.195616 24592 raft_consensus.cc:2272] T f0fd88947c65482b8e9d6b23e96a4db2 P 07a5248f4bf84ded832b7e261c3768b1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:47.197293 24592 tablet_replica.cc:333] T 9052c9d021c5487da14be02e7852fb2a P 07a5248f4bf84ded832b7e261c3768b1: stopping tablet replica
I20260807 01:45:47.197829 24592 raft_consensus.cc:2243] T 9052c9d021c5487da14be02e7852fb2a P 07a5248f4bf84ded832b7e261c3768b1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 01:45:47.198385 24592 raft_consensus.cc:2272] T 9052c9d021c5487da14be02e7852fb2a P 07a5248f4bf84ded832b7e261c3768b1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:47.226418 24592 tablet_server.cc:196] TabletServer@127.24.4.3:0 shutdown complete.
I20260807 01:45:47.247361 24592 master.cc:562] Master@127.24.4.62:36655 shutting down...
I20260807 01:45:47.562394 24592 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e [term 1 LEADER]: Raft consensus shutting down.
I20260807 01:45:47.563047 24592 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 01:45:47.563416 24592 tablet_replica.cc:333] T 00000000000000000000000000000000 P ec32210d703d4425858186c0b0c1d32e: stopping tablet replica
I20260807 01:45:47.585319 24592 master.cc:584] Master@127.24.4.62:36655 shutdown complete.
07a5248f4bf84ded832b7e261c3768b1 1
9cb56177a511438f9f4a8c9ad5d64852 4
f22c8ad8eba94330950285be7e2a09a8 4
07a5248f4bf84ded832b7e261c3768b1 3
9cb56177a511438f9f4a8c9ad5d64852 3
f22c8ad8eba94330950285be7e2a09a8 3
07a5248f4bf84ded832b7e261c3768b1 1
9cb56177a511438f9f4a8c9ad5d64852 0
f22c8ad8eba94330950285be7e2a09a8 8
07a5248f4bf84ded832b7e261c3768b1 3
9cb56177a511438f9f4a8c9ad5d64852 3
f22c8ad8eba94330950285be7e2a09a8 3
[ OK ] LeaderRebalancerTest.FunctionalTestForDivided (34611 ms)
[ RUN ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260807 01:45:47.636523 24592 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.24.4.62:42161
I20260807 01:45:47.637835 24592 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 01:45:47.644392 25434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 01:45:47.645258 25433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 01:45:47.645865 25436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:45:47.646433 24592 server_base.cc:1061] running on GCE node
I20260807 01:45:47.647543 24592 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 01:45:47.647758 24592 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 01:45:47.647876 24592 hybrid_clock.cc:648] HybridClock initialized: now 1786067147647864 us; error 0 us; skew 500 ppm
I20260807 01:45:47.648346 24592 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 01:45:47.651960 24592 webserver.cc:533] Webserver started at http://127.24.4.62:38163/ using document root <none> and password file <none>
I20260807 01:45:47.652601 24592 fs_manager.cc:362] Metadata directory not provided
I20260807 01:45:47.652810 24592 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 01:45:47.653061 24592 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 01:45:47.654628 24592 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786067112798834-24592-0/minicluster-data/master-0-root/instance:
uuid: "42fc07fefb5b442bafb484e48a00f8d3"
format_stamp: "Formatted at 2026-08-07 01:45:47 on dist-test-slave-zj8q"
I20260807 01:45:47.659869 24592 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 01:45:47.664566 25448 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 01:45:47.665453 24592 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 01:45:47.665803 24592 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786067112798834-24592-0/minicluster-data/master-0-root
uuid: "42fc07fefb5b442bafb484e48a00f8d3"
format_stamp: "Formatted at 2026-08-07 01:45:47 on dist-test-slave-zj8q"
I20260807 01:45:47.666108 24592 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786067112798834-24592-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786067112798834-24592-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786067112798834-24592-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 01:45:47.697002 24592 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 01:45:47.698428 24592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 01:45:47.742130 24592 rpc_server.cc:307] RPC server started. Bound to: 127.24.4.62:42161
I20260807 01:45:47.742336 25519 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.4.62:42161 every 8 connection(s)
I20260807 01:45:47.746308 25520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:47.756750 25520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3: Bootstrap starting.
I20260807 01:45:47.761736 25520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:47.766546 25520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3: No bootstrap required, opened a new log
I20260807 01:45:47.769062 25520 raft_consensus.cc:359] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42fc07fefb5b442bafb484e48a00f8d3" member_type: VOTER }
I20260807 01:45:47.769553 25520 raft_consensus.cc:385] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:47.769802 25520 raft_consensus.cc:740] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42fc07fefb5b442bafb484e48a00f8d3, State: Initialized, Role: FOLLOWER
I20260807 01:45:47.770408 25520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42fc07fefb5b442bafb484e48a00f8d3" member_type: VOTER }
I20260807 01:45:47.770869 25520 raft_consensus.cc:399] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 01:45:47.771116 25520 raft_consensus.cc:493] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 01:45:47.771404 25520 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:47.775691 25520 raft_consensus.cc:515] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42fc07fefb5b442bafb484e48a00f8d3" member_type: VOTER }
I20260807 01:45:47.776266 25520 leader_election.cc:304] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42fc07fefb5b442bafb484e48a00f8d3; no voters:
I20260807 01:45:47.777482 25520 leader_election.cc:290] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 01:45:47.777915 25524 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:45:47.779353 25524 raft_consensus.cc:697] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3 [term 1 LEADER]: Becoming Leader. State: Replica: 42fc07fefb5b442bafb484e48a00f8d3, State: Running, Role: LEADER
I20260807 01:45:47.780113 25524 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42fc07fefb5b442bafb484e48a00f8d3" member_type: VOTER }
I20260807 01:45:47.780375 25520 sys_catalog.cc:565] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 01:45:47.785292 25525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42fc07fefb5b442bafb484e48a00f8d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42fc07fefb5b442bafb484e48a00f8d3" member_type: VOTER } }
I20260807 01:45:47.785540 25526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42fc07fefb5b442bafb484e48a00f8d3. Latest consensus state: current_term: 1 leader_uuid: "42fc07fefb5b442bafb484e48a00f8d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42fc07fefb5b442bafb484e48a00f8d3" member_type: VOTER } }
I20260807 01:45:47.786238 25525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3 [sys.catalog]: This master's current role is: LEADER
I20260807 01:45:47.786284 25526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3 [sys.catalog]: This master's current role is: LEADER
I20260807 01:45:47.789227 25536 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260807 01:45:47.794677 25536 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260807 01:45:47.798810 24592 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 01:45:47.806254 25536 catalog_manager.cc:1383] Generated new cluster ID: cc10daab3a684c479c88c7e228d227ef
I20260807 01:45:47.806573 25536 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260807 01:45:47.822576 25536 catalog_manager.cc:1406] Generated new certificate authority record
I20260807 01:45:47.823915 25536 catalog_manager.cc:1540] Loading token signing keys...
I20260807 01:45:47.837046 25536 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3: Generated new TSK 0
I20260807 01:45:47.837682 25536 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260807 01:45:47.870338 24592 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 01:45:47.877655 25555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 01:45:47.878602 25557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:45:47.882270 24592 server_base.cc:1061] running on GCE node
W20260807 01:45:47.882236 25560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:45:47.883635 24592 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 01:45:47.883841 24592 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 01:45:47.883965 24592 hybrid_clock.cc:648] HybridClock initialized: now 1786067147883953 us; error 0 us; skew 500 ppm
I20260807 01:45:47.884450 24592 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 01:45:47.887245 24592 webserver.cc:533] Webserver started at http://127.24.4.1:36115/ using document root <none> and password file <none>
I20260807 01:45:47.887763 24592 fs_manager.cc:362] Metadata directory not provided
I20260807 01:45:47.887928 24592 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 01:45:47.888146 24592 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 01:45:47.889228 24592 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786067112798834-24592-0/minicluster-data/ts-0-root/instance:
uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
format_stamp: "Formatted at 2026-08-07 01:45:47 on dist-test-slave-zj8q"
I20260807 01:45:47.894593 24592 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 01:45:47.899051 25570 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 01:45:47.900067 24592 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 01:45:47.900408 24592 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786067112798834-24592-0/minicluster-data/ts-0-root
uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
format_stamp: "Formatted at 2026-08-07 01:45:47 on dist-test-slave-zj8q"
I20260807 01:45:47.900697 24592 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786067112798834-24592-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786067112798834-24592-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786067112798834-24592-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 01:45:47.913741 24592 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 01:45:47.915205 24592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 01:45:47.916847 24592 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 01:45:47.920382 24592 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 01:45:47.920626 24592 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 01:45:47.920894 24592 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 01:45:47.921051 24592 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 01:45:47.970304 24592 rpc_server.cc:307] RPC server started. Bound to: 127.24.4.1:33955
I20260807 01:45:47.970414 25655 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.4.1:33955 every 8 connection(s)
I20260807 01:45:47.976353 24592 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 01:45:47.983853 25664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 01:45:47.985231 25667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:45:47.988634 24592 server_base.cc:1061] running on GCE node
W20260807 01:45:47.989009 25669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:45:47.991534 24592 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 01:45:47.991806 24592 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 01:45:47.991987 24592 hybrid_clock.cc:648] HybridClock initialized: now 1786067147991967 us; error 0 us; skew 500 ppm
I20260807 01:45:47.992609 24592 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 01:45:47.994788 25656 heartbeater.cc:344] Connected to a master server at 127.24.4.62:42161
I20260807 01:45:47.995169 25656 heartbeater.cc:461] Registering TS with master...
I20260807 01:45:47.995962 25656 heartbeater.cc:507] Master 127.24.4.62:42161 requested a full tablet report, sending...
I20260807 01:45:47.996311 24592 webserver.cc:533] Webserver started at http://127.24.4.2:35245/ using document root <none> and password file <none>
I20260807 01:45:47.996860 24592 fs_manager.cc:362] Metadata directory not provided
I20260807 01:45:47.997059 24592 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 01:45:47.997433 24592 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 01:45:47.998507 25476 ts_manager.cc:194] Registered new tserver with Master: 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955)
I20260807 01:45:47.998991 24592 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786067112798834-24592-0/minicluster-data/ts-1-root/instance:
uuid: "8f9c53697dc142e4af11d5f5e0e167de"
format_stamp: "Formatted at 2026-08-07 01:45:47 on dist-test-slave-zj8q"
I20260807 01:45:48.000468 25476 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59252
I20260807 01:45:48.004053 24592 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 01:45:48.007817 25678 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 01:45:48.008677 24592 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 01:45:48.008985 24592 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786067112798834-24592-0/minicluster-data/ts-1-root
uuid: "8f9c53697dc142e4af11d5f5e0e167de"
format_stamp: "Formatted at 2026-08-07 01:45:47 on dist-test-slave-zj8q"
I20260807 01:45:48.009269 24592 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786067112798834-24592-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786067112798834-24592-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786067112798834-24592-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 01:45:48.036468 24592 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 01:45:48.038050 24592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 01:45:48.039892 24592 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 01:45:48.042549 24592 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 01:45:48.042802 24592 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 01:45:48.043051 24592 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 01:45:48.043210 24592 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 01:45:48.093077 24592 rpc_server.cc:307] RPC server started. Bound to: 127.24.4.2:36387
I20260807 01:45:48.093211 25778 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.4.2:36387 every 8 connection(s)
I20260807 01:45:48.098282 24592 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 01:45:48.105800 25782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 01:45:48.107286 25784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:45:48.109409 24592 server_base.cc:1061] running on GCE node
W20260807 01:45:48.110692 25790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:45:48.111032 25779 heartbeater.cc:344] Connected to a master server at 127.24.4.62:42161
I20260807 01:45:48.111471 25779 heartbeater.cc:461] Registering TS with master...
I20260807 01:45:48.111837 24592 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 01:45:48.112066 24592 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 01:45:48.112244 24592 hybrid_clock.cc:648] HybridClock initialized: now 1786067148112228 us; error 0 us; skew 500 ppm
I20260807 01:45:48.112394 25779 heartbeater.cc:507] Master 127.24.4.62:42161 requested a full tablet report, sending...
I20260807 01:45:48.112859 24592 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 01:45:48.114854 25476 ts_manager.cc:194] Registered new tserver with Master: 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387)
I20260807 01:45:48.116403 24592 webserver.cc:533] Webserver started at http://127.24.4.3:44165/ using document root <none> and password file <none>
I20260807 01:45:48.116482 25476 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59268
I20260807 01:45:48.117067 24592 fs_manager.cc:362] Metadata directory not provided
I20260807 01:45:48.117307 24592 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 01:45:48.117672 24592 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 01:45:48.119011 24592 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786067112798834-24592-0/minicluster-data/ts-2-root/instance:
uuid: "b147ae01363d411980d3a99e72915a00"
format_stamp: "Formatted at 2026-08-07 01:45:48 on dist-test-slave-zj8q"
I20260807 01:45:48.124264 24592 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 01:45:48.128196 25795 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 01:45:48.129050 24592 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 01:45:48.129354 24592 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786067112798834-24592-0/minicluster-data/ts-2-root
uuid: "b147ae01363d411980d3a99e72915a00"
format_stamp: "Formatted at 2026-08-07 01:45:48 on dist-test-slave-zj8q"
I20260807 01:45:48.129678 24592 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786067112798834-24592-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786067112798834-24592-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786067112798834-24592-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 01:45:48.170378 24592 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 01:45:48.171717 24592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 01:45:48.173249 24592 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 01:45:48.175724 24592 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 01:45:48.175945 24592 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 01:45:48.176198 24592 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 01:45:48.176347 24592 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 01:45:48.226208 24592 rpc_server.cc:307] RPC server started. Bound to: 127.24.4.3:39743
I20260807 01:45:48.226312 25897 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.4.3:39743 every 8 connection(s)
I20260807 01:45:48.243891 25899 heartbeater.cc:344] Connected to a master server at 127.24.4.62:42161
I20260807 01:45:48.244374 25899 heartbeater.cc:461] Registering TS with master...
I20260807 01:45:48.245357 25899 heartbeater.cc:507] Master 127.24.4.62:42161 requested a full tablet report, sending...
I20260807 01:45:48.247895 25476 ts_manager.cc:194] Registered new tserver with Master: b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:48.248131 24592 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018681867s
I20260807 01:45:48.249424 25476 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59280
I20260807 01:45:48.279635 25476 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59282:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 01:45:48.283155 25476 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 01:45:48.362804 25611 tablet_service.cc:1511] Processing CreateTablet for tablet 06fd48ba8b4b49e2a0f51f1f9b8a67df (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260807 01:45:48.366245 25611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06fd48ba8b4b49e2a0f51f1f9b8a67df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.366290 25612 tablet_service.cc:1511] Processing CreateTablet for tablet a8ebd958223c4616b2055f66cfcb5581 (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260807 01:45:48.368674 25612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8ebd958223c4616b2055f66cfcb5581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.373709 25610 tablet_service.cc:1511] Processing CreateTablet for tablet fde466361a8d40a48394033c480ecf50 (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260807 01:45:48.375377 25610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fde466361a8d40a48394033c480ecf50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.395874 25609 tablet_service.cc:1511] Processing CreateTablet for tablet 5a7b6bccb54c42b4895fdfdc6230eea9 (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260807 01:45:48.397361 25609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a7b6bccb54c42b4895fdfdc6230eea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.398195 25606 tablet_service.cc:1511] Processing CreateTablet for tablet 39269a566fe848f481ab0b1a50d48220 (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260807 01:45:48.398412 25607 tablet_service.cc:1511] Processing CreateTablet for tablet a7a55cd1670e40ec94e4269dc15517bb (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260807 01:45:48.399683 25606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39269a566fe848f481ab0b1a50d48220. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.400354 25607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7a55cd1670e40ec94e4269dc15517bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.404230 25612 tablet_service.cc:1511] Processing CreateTablet for tablet 75e20a485954424c8b721f0ce6216da2 (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260807 01:45:48.405680 25612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75e20a485954424c8b721f0ce6216da2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.409433 25605 tablet_service.cc:1511] Processing CreateTablet for tablet 061b319309704115ac6bc2716e08fd4a (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260807 01:45:48.411022 25605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 061b319309704115ac6bc2716e08fd4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.413163 25839 tablet_service.cc:1511] Processing CreateTablet for tablet a8ebd958223c4616b2055f66cfcb5581 (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260807 01:45:48.414685 25839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8ebd958223c4616b2055f66cfcb5581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.415642 25828 tablet_service.cc:1511] Processing CreateTablet for tablet 75e20a485954424c8b721f0ce6216da2 (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260807 01:45:48.417285 25602 tablet_service.cc:1511] Processing CreateTablet for tablet eff0dbafbd3f457ba6c59d0e8bf16705 (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260807 01:45:48.418864 25602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eff0dbafbd3f457ba6c59d0e8bf16705. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.418779 25827 tablet_service.cc:1511] Processing CreateTablet for tablet 061b319309704115ac6bc2716e08fd4a (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260807 01:45:48.419476 25838 tablet_service.cc:1511] Processing CreateTablet for tablet 06fd48ba8b4b49e2a0f51f1f9b8a67df (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260807 01:45:48.417179 25828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75e20a485954424c8b721f0ce6216da2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.420871 25838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06fd48ba8b4b49e2a0f51f1f9b8a67df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.421535 25827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 061b319309704115ac6bc2716e08fd4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.423489 25837 tablet_service.cc:1511] Processing CreateTablet for tablet fde466361a8d40a48394033c480ecf50 (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260807 01:45:48.424983 25837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fde466361a8d40a48394033c480ecf50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.427533 25834 tablet_service.cc:1511] Processing CreateTablet for tablet 5a7b6bccb54c42b4895fdfdc6230eea9 (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260807 01:45:48.429051 25834 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a7b6bccb54c42b4895fdfdc6230eea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.431605 25832 tablet_service.cc:1511] Processing CreateTablet for tablet a7a55cd1670e40ec94e4269dc15517bb (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260807 01:45:48.433086 25832 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7a55cd1670e40ec94e4269dc15517bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.435457 25830 tablet_service.cc:1511] Processing CreateTablet for tablet 39269a566fe848f481ab0b1a50d48220 (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260807 01:45:48.436925 25830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39269a566fe848f481ab0b1a50d48220. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.440893 25826 tablet_service.cc:1511] Processing CreateTablet for tablet eff0dbafbd3f457ba6c59d0e8bf16705 (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260807 01:45:48.442426 25826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eff0dbafbd3f457ba6c59d0e8bf16705. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.443907 25725 tablet_service.cc:1511] Processing CreateTablet for tablet a8ebd958223c4616b2055f66cfcb5581 (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260807 01:45:48.445418 25725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8ebd958223c4616b2055f66cfcb5581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.451749 25724 tablet_service.cc:1511] Processing CreateTablet for tablet 06fd48ba8b4b49e2a0f51f1f9b8a67df (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260807 01:45:48.453245 25724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06fd48ba8b4b49e2a0f51f1f9b8a67df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.454483 25607 tablet_service.cc:1511] Processing CreateTablet for tablet 31f1ba5ca06f461a8b3001ca5c4c48a3 (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260807 01:45:48.456892 25607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31f1ba5ca06f461a8b3001ca5c4c48a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.458302 25723 tablet_service.cc:1511] Processing CreateTablet for tablet fde466361a8d40a48394033c480ecf50 (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260807 01:45:48.459771 25723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fde466361a8d40a48394033c480ecf50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.471302 25923 tablet_bootstrap.cc:492] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00: Bootstrap starting.
I20260807 01:45:48.477901 25923 tablet_bootstrap.cc:654] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:48.483131 25722 tablet_service.cc:1511] Processing CreateTablet for tablet 5a7b6bccb54c42b4895fdfdc6230eea9 (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260807 01:45:48.484791 25722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a7b6bccb54c42b4895fdfdc6230eea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.492067 25827 tablet_service.cc:1511] Processing CreateTablet for tablet 31f1ba5ca06f461a8b3001ca5c4c48a3 (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260807 01:45:48.493712 25827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31f1ba5ca06f461a8b3001ca5c4c48a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.492115 25721 tablet_service.cc:1511] Processing CreateTablet for tablet a7a55cd1670e40ec94e4269dc15517bb (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260807 01:45:48.494694 25719 tablet_service.cc:1511] Processing CreateTablet for tablet 39269a566fe848f481ab0b1a50d48220 (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260807 01:45:48.496133 25721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7a55cd1670e40ec94e4269dc15517bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.496795 25719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39269a566fe848f481ab0b1a50d48220. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.502199 25712 tablet_service.cc:1511] Processing CreateTablet for tablet 75e20a485954424c8b721f0ce6216da2 (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260807 01:45:48.503738 25712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75e20a485954424c8b721f0ce6216da2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.507620 25709 tablet_service.cc:1511] Processing CreateTablet for tablet 061b319309704115ac6bc2716e08fd4a (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260807 01:45:48.509152 25709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 061b319309704115ac6bc2716e08fd4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.511513 25708 tablet_service.cc:1511] Processing CreateTablet for tablet eff0dbafbd3f457ba6c59d0e8bf16705 (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260807 01:45:48.513401 25708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eff0dbafbd3f457ba6c59d0e8bf16705. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.516425 25923 tablet_bootstrap.cc:492] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00: No bootstrap required, opened a new log
I20260807 01:45:48.516120 25707 tablet_service.cc:1511] Processing CreateTablet for tablet 31f1ba5ca06f461a8b3001ca5c4c48a3 (DEFAULT_TABLE table=test-workload [id=135f338532a94fd8b5599fbda4d3afb2]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260807 01:45:48.516995 25923 ts_tablet_manager.cc:1403] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00: Time spent bootstrapping tablet: real 0.046s user 0.012s sys 0.007s
I20260807 01:45:48.517616 25707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31f1ba5ca06f461a8b3001ca5c4c48a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:45:48.520022 25923 raft_consensus.cc:359] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.520813 25923 raft_consensus.cc:385] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:48.521123 25923 raft_consensus.cc:740] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b147ae01363d411980d3a99e72915a00, State: Initialized, Role: FOLLOWER
I20260807 01:45:48.530526 25923 consensus_queue.cc:260] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.586700 25922 tablet_bootstrap.cc:492] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8: Bootstrap starting.
I20260807 01:45:48.586705 25899 heartbeater.cc:499] Master 127.24.4.62:42161 was elected leader, sending a full tablet report...
I20260807 01:45:48.587445 25927 tablet_bootstrap.cc:492] T a7a55cd1670e40ec94e4269dc15517bb P 8f9c53697dc142e4af11d5f5e0e167de: Bootstrap starting.
I20260807 01:45:48.594774 25922 tablet_bootstrap.cc:654] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:48.598367 25923 ts_tablet_manager.cc:1434] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00: Time spent starting tablet: real 0.081s user 0.014s sys 0.012s
I20260807 01:45:48.599473 25923 tablet_bootstrap.cc:492] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00: Bootstrap starting.
I20260807 01:45:48.602648 25927 tablet_bootstrap.cc:654] T a7a55cd1670e40ec94e4269dc15517bb P 8f9c53697dc142e4af11d5f5e0e167de: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:48.604808 25922 tablet_bootstrap.cc:492] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8: No bootstrap required, opened a new log
I20260807 01:45:48.605350 25922 ts_tablet_manager.cc:1403] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8: Time spent bootstrapping tablet: real 0.019s user 0.012s sys 0.000s
I20260807 01:45:48.608733 25922 raft_consensus.cc:359] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.609792 25922 raft_consensus.cc:385] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:48.610255 25922 raft_consensus.cc:740] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Initialized, Role: FOLLOWER
I20260807 01:45:48.611299 25923 tablet_bootstrap.cc:654] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:48.611655 25922 consensus_queue.cc:260] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.622102 25922 ts_tablet_manager.cc:1434] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8: Time spent starting tablet: real 0.016s user 0.008s sys 0.009s
I20260807 01:45:48.622212 25656 heartbeater.cc:499] Master 127.24.4.62:42161 was elected leader, sending a full tablet report...
I20260807 01:45:48.625754 25927 tablet_bootstrap.cc:492] T a7a55cd1670e40ec94e4269dc15517bb P 8f9c53697dc142e4af11d5f5e0e167de: No bootstrap required, opened a new log
I20260807 01:45:48.626667 25927 ts_tablet_manager.cc:1403] T a7a55cd1670e40ec94e4269dc15517bb P 8f9c53697dc142e4af11d5f5e0e167de: Time spent bootstrapping tablet: real 0.040s user 0.019s sys 0.010s
I20260807 01:45:48.631139 25927 raft_consensus.cc:359] T a7a55cd1670e40ec94e4269dc15517bb P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.632412 25927 raft_consensus.cc:385] T a7a55cd1670e40ec94e4269dc15517bb P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:48.632896 25927 raft_consensus.cc:740] T a7a55cd1670e40ec94e4269dc15517bb P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f9c53697dc142e4af11d5f5e0e167de, State: Initialized, Role: FOLLOWER
I20260807 01:45:48.633807 25923 tablet_bootstrap.cc:492] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00: No bootstrap required, opened a new log
I20260807 01:45:48.634579 25923 ts_tablet_manager.cc:1403] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00: Time spent bootstrapping tablet: real 0.035s user 0.018s sys 0.001s
I20260807 01:45:48.634369 25927 consensus_queue.cc:260] T a7a55cd1670e40ec94e4269dc15517bb P 8f9c53697dc142e4af11d5f5e0e167de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.637626 25779 heartbeater.cc:499] Master 127.24.4.62:42161 was elected leader, sending a full tablet report...
I20260807 01:45:48.638434 25927 ts_tablet_manager.cc:1434] T a7a55cd1670e40ec94e4269dc15517bb P 8f9c53697dc142e4af11d5f5e0e167de: Time spent starting tablet: real 0.011s user 0.008s sys 0.002s
I20260807 01:45:48.638732 25923 raft_consensus.cc:359] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.639914 25923 raft_consensus.cc:385] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:48.640614 25923 raft_consensus.cc:740] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b147ae01363d411980d3a99e72915a00, State: Initialized, Role: FOLLOWER
I20260807 01:45:48.640810 25927 tablet_bootstrap.cc:492] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de: Bootstrap starting.
I20260807 01:45:48.642203 25922 tablet_bootstrap.cc:492] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8: Bootstrap starting.
I20260807 01:45:48.649288 25927 tablet_bootstrap.cc:654] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:48.649628 25923 consensus_queue.cc:260] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.652580 25923 ts_tablet_manager.cc:1434] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00: Time spent starting tablet: real 0.018s user 0.008s sys 0.000s
I20260807 01:45:48.654145 25923 tablet_bootstrap.cc:492] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00: Bootstrap starting.
I20260807 01:45:48.659672 25922 tablet_bootstrap.cc:654] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:48.660260 25927 tablet_bootstrap.cc:492] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de: No bootstrap required, opened a new log
I20260807 01:45:48.660804 25927 ts_tablet_manager.cc:1403] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de: Time spent bootstrapping tablet: real 0.021s user 0.012s sys 0.001s
I20260807 01:45:48.661465 25923 tablet_bootstrap.cc:654] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:48.663960 25927 raft_consensus.cc:359] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.664739 25927 raft_consensus.cc:385] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:48.665053 25927 raft_consensus.cc:740] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f9c53697dc142e4af11d5f5e0e167de, State: Initialized, Role: FOLLOWER
I20260807 01:45:48.665970 25927 consensus_queue.cc:260] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.668618 25927 ts_tablet_manager.cc:1434] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de: Time spent starting tablet: real 0.007s user 0.009s sys 0.000s
I20260807 01:45:48.669649 25927 tablet_bootstrap.cc:492] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de: Bootstrap starting.
I20260807 01:45:48.675324 25927 tablet_bootstrap.cc:654] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:48.681419 25927 tablet_bootstrap.cc:492] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de: No bootstrap required, opened a new log
I20260807 01:45:48.682093 25927 ts_tablet_manager.cc:1403] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.011s
I20260807 01:45:48.685045 25927 raft_consensus.cc:359] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.685855 25927 raft_consensus.cc:385] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:48.686182 25927 raft_consensus.cc:740] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f9c53697dc142e4af11d5f5e0e167de, State: Initialized, Role: FOLLOWER
I20260807 01:45:48.687064 25927 consensus_queue.cc:260] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.689623 25922 tablet_bootstrap.cc:492] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8: No bootstrap required, opened a new log
I20260807 01:45:48.689687 25927 ts_tablet_manager.cc:1434] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260807 01:45:48.690209 25923 tablet_bootstrap.cc:492] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00: No bootstrap required, opened a new log
I20260807 01:45:48.690343 25922 ts_tablet_manager.cc:1403] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8: Time spent bootstrapping tablet: real 0.048s user 0.010s sys 0.012s
I20260807 01:45:48.690696 25923 ts_tablet_manager.cc:1403] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00: Time spent bootstrapping tablet: real 0.037s user 0.015s sys 0.020s
I20260807 01:45:48.690949 25927 tablet_bootstrap.cc:492] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 8f9c53697dc142e4af11d5f5e0e167de: Bootstrap starting.
I20260807 01:45:48.694185 25922 raft_consensus.cc:359] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.694278 25923 raft_consensus.cc:359] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.695001 25922 raft_consensus.cc:385] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:48.695394 25923 raft_consensus.cc:385] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:48.695466 25922 raft_consensus.cc:740] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Initialized, Role: FOLLOWER
I20260807 01:45:48.695832 25923 raft_consensus.cc:740] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b147ae01363d411980d3a99e72915a00, State: Initialized, Role: FOLLOWER
I20260807 01:45:48.696727 25922 consensus_queue.cc:260] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.696851 25923 consensus_queue.cc:260] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.698477 25927 tablet_bootstrap.cc:654] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 8f9c53697dc142e4af11d5f5e0e167de: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:48.699206 25923 ts_tablet_manager.cc:1434] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260807 01:45:48.700278 25923 tablet_bootstrap.cc:492] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00: Bootstrap starting.
I20260807 01:45:48.707285 25923 tablet_bootstrap.cc:654] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:48.707479 25922 ts_tablet_manager.cc:1434] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8: Time spent starting tablet: real 0.017s user 0.007s sys 0.000s
I20260807 01:45:48.708554 25922 tablet_bootstrap.cc:492] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8: Bootstrap starting.
I20260807 01:45:48.717636 25922 tablet_bootstrap.cc:654] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:48.719419 25927 tablet_bootstrap.cc:492] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 8f9c53697dc142e4af11d5f5e0e167de: No bootstrap required, opened a new log
I20260807 01:45:48.719967 25927 ts_tablet_manager.cc:1403] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 8f9c53697dc142e4af11d5f5e0e167de: Time spent bootstrapping tablet: real 0.029s user 0.010s sys 0.017s
I20260807 01:45:48.720714 25923 tablet_bootstrap.cc:492] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00: No bootstrap required, opened a new log
I20260807 01:45:48.721216 25923 ts_tablet_manager.cc:1403] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00: Time spent bootstrapping tablet: real 0.021s user 0.013s sys 0.003s
I20260807 01:45:48.722887 25922 tablet_bootstrap.cc:492] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8: No bootstrap required, opened a new log
I20260807 01:45:48.723425 25922 ts_tablet_manager.cc:1403] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8: Time spent bootstrapping tablet: real 0.015s user 0.012s sys 0.001s
I20260807 01:45:48.723073 25927 raft_consensus.cc:359] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.723835 25927 raft_consensus.cc:385] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:48.724157 25927 raft_consensus.cc:740] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f9c53697dc142e4af11d5f5e0e167de, State: Initialized, Role: FOLLOWER
I20260807 01:45:48.724104 25923 raft_consensus.cc:359] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.724885 25923 raft_consensus.cc:385] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:48.725203 25923 raft_consensus.cc:740] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b147ae01363d411980d3a99e72915a00, State: Initialized, Role: FOLLOWER
I20260807 01:45:48.725040 25927 consensus_queue.cc:260] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 8f9c53697dc142e4af11d5f5e0e167de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
W20260807 01:45:48.725818 25660 tablet.cc:2406] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 01:45:48.726191 25923 consensus_queue.cc:260] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.727375 25927 ts_tablet_manager.cc:1434] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 8f9c53697dc142e4af11d5f5e0e167de: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 01:45:48.728400 25927 tablet_bootstrap.cc:492] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de: Bootstrap starting.
I20260807 01:45:48.728364 25923 ts_tablet_manager.cc:1434] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 01:45:48.729518 25923 tablet_bootstrap.cc:492] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00: Bootstrap starting.
I20260807 01:45:48.725819 25922 raft_consensus.cc:359] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.730446 25922 raft_consensus.cc:385] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:48.730765 25922 raft_consensus.cc:740] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Initialized, Role: FOLLOWER
I20260807 01:45:48.731649 25922 consensus_queue.cc:260] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.734079 25922 ts_tablet_manager.cc:1434] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8: Time spent starting tablet: real 0.010s user 0.001s sys 0.003s
I20260807 01:45:48.736140 25927 tablet_bootstrap.cc:654] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:48.737986 25923 tablet_bootstrap.cc:654] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:48.737874 25922 tablet_bootstrap.cc:492] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8: Bootstrap starting.
I20260807 01:45:48.743908 25926 raft_consensus.cc:493] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:48.744617 25926 raft_consensus.cc:515] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.747511 25922 tablet_bootstrap.cc:654] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:48.753039 25947 raft_consensus.cc:493] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:48.753638 25947 raft_consensus.cc:515] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.756541 25947 leader_election.cc:290] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387)
I20260807 01:45:48.766088 25926 leader_election.cc:290] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387)
I20260807 01:45:48.769246 25923 tablet_bootstrap.cc:492] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00: No bootstrap required, opened a new log
I20260807 01:45:48.776140 25923 ts_tablet_manager.cc:1403] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00: Time spent bootstrapping tablet: real 0.047s user 0.009s sys 0.008s
I20260807 01:45:48.776170 25922 tablet_bootstrap.cc:492] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8: No bootstrap required, opened a new log
I20260807 01:45:48.778330 25922 ts_tablet_manager.cc:1403] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8: Time spent bootstrapping tablet: real 0.043s user 0.011s sys 0.023s
I20260807 01:45:48.779253 25923 raft_consensus.cc:359] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.780011 25923 raft_consensus.cc:385] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:48.780332 25923 raft_consensus.cc:740] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b147ae01363d411980d3a99e72915a00, State: Initialized, Role: FOLLOWER
I20260807 01:45:48.781692 25922 raft_consensus.cc:359] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.782477 25923 consensus_queue.cc:260] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.783787 25922 raft_consensus.cc:385] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:48.784289 25922 raft_consensus.cc:740] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Initialized, Role: FOLLOWER
I20260807 01:45:48.785161 25922 consensus_queue.cc:260] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.785782 25923 ts_tablet_manager.cc:1434] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00: Time spent starting tablet: real 0.009s user 0.005s sys 0.001s
I20260807 01:45:48.786935 25923 tablet_bootstrap.cc:492] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00: Bootstrap starting.
I20260807 01:45:48.787472 25931 raft_consensus.cc:493] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:48.787701 25922 ts_tablet_manager.cc:1434] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8: Time spent starting tablet: real 0.009s user 0.006s sys 0.001s
I20260807 01:45:48.787949 25931 raft_consensus.cc:515] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.788980 25922 tablet_bootstrap.cc:492] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8: Bootstrap starting.
I20260807 01:45:48.791006 25737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061b319309704115ac6bc2716e08fd4a" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de" is_pre_election: true
I20260807 01:45:48.791646 25927 tablet_bootstrap.cc:492] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de: No bootstrap required, opened a new log
I20260807 01:45:48.792222 25927 ts_tablet_manager.cc:1403] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de: Time spent bootstrapping tablet: real 0.064s user 0.019s sys 0.025s
I20260807 01:45:48.792618 25735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e20a485954424c8b721f0ce6216da2" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de" is_pre_election: true
W20260807 01:45:48.794119 25541 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 01:45:48.795650 25927 raft_consensus.cc:359] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.796825 25927 raft_consensus.cc:385] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260807 01:45:48.796918 25799 leader_election.cc:343] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:48.797201 25927 raft_consensus.cc:740] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f9c53697dc142e4af11d5f5e0e167de, State: Initialized, Role: FOLLOWER
I20260807 01:45:48.797669 25922 tablet_bootstrap.cc:654] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:48.798110 25927 consensus_queue.cc:260] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.799167 25735 raft_consensus.cc:2468] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 0.
I20260807 01:45:48.802793 25927 ts_tablet_manager.cc:1434] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de: Time spent starting tablet: real 0.010s user 0.002s sys 0.006s
I20260807 01:45:48.806396 25922 tablet_bootstrap.cc:492] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8: No bootstrap required, opened a new log
I20260807 01:45:48.806545 25799 leader_election.cc:304] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f9c53697dc142e4af11d5f5e0e167de, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:45:48.807173 25922 ts_tablet_manager.cc:1403] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.004s
I20260807 01:45:48.807540 25926 raft_consensus.cc:2804] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:45:48.807950 25926 raft_consensus.cc:493] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:45:48.808302 25926 raft_consensus.cc:3060] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:48.811144 25931 leader_election.cc:290] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:48.810768 25922 raft_consensus.cc:359] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.811611 25922 raft_consensus.cc:385] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:48.811956 25922 raft_consensus.cc:740] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Initialized, Role: FOLLOWER
I20260807 01:45:48.812718 25922 consensus_queue.cc:260] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.815196 25922 ts_tablet_manager.cc:1434] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260807 01:45:48.816365 25922 tablet_bootstrap.cc:492] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8: Bootstrap starting.
I20260807 01:45:48.816187 25926 raft_consensus.cc:515] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.821141 25923 tablet_bootstrap.cc:654] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:48.823455 25922 tablet_bootstrap.cc:654] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:48.824373 25926 leader_election.cc:290] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 election: Requested vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387)
I20260807 01:45:48.827989 25735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e20a485954424c8b721f0ce6216da2" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:45:48.828706 25735 raft_consensus.cc:3060] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:48.829911 25927 tablet_bootstrap.cc:492] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de: Bootstrap starting.
I20260807 01:45:48.836313 25923 tablet_bootstrap.cc:492] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00: No bootstrap required, opened a new log
I20260807 01:45:48.839169 25735 raft_consensus.cc:2468] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 1.
I20260807 01:45:48.840814 25799 leader_election.cc:304] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f9c53697dc142e4af11d5f5e0e167de, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:45:48.841936 25926 raft_consensus.cc:2804] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:45:48.842533 25926 raft_consensus.cc:697] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 1 LEADER]: Becoming Leader. State: Replica: b147ae01363d411980d3a99e72915a00, State: Running, Role: LEADER
I20260807 01:45:48.843674 25933 raft_consensus.cc:493] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:48.843423 25926 consensus_queue.cc:237] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.844345 25927 tablet_bootstrap.cc:654] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:48.844201 25933 raft_consensus.cc:515] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.850926 25923 ts_tablet_manager.cc:1403] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00: Time spent bootstrapping tablet: real 0.064s user 0.012s sys 0.001s
I20260807 01:45:48.855722 25933 leader_election.cc:290] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:48.883807 25922 tablet_bootstrap.cc:492] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8: No bootstrap required, opened a new log
I20260807 01:45:48.884781 25922 ts_tablet_manager.cc:1403] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8: Time spent bootstrapping tablet: real 0.069s user 0.016s sys 0.016s
I20260807 01:45:48.883845 25476 catalog_manager.cc:5697] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 reported cstate change: term changed from 0 to 1, leader changed from <none> to b147ae01363d411980d3a99e72915a00 (127.24.4.3). New cstate: current_term: 1 leader_uuid: "b147ae01363d411980d3a99e72915a00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: HEALTHY } } }
I20260807 01:45:48.890350 25922 raft_consensus.cc:359] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.890689 25624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061b319309704115ac6bc2716e08fd4a" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" is_pre_election: true
I20260807 01:45:48.883653 25923 raft_consensus.cc:359] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.891343 25922 raft_consensus.cc:385] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:48.890648 25952 raft_consensus.cc:493] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:48.892210 25922 raft_consensus.cc:740] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Initialized, Role: FOLLOWER
I20260807 01:45:48.892292 25927 tablet_bootstrap.cc:492] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de: No bootstrap required, opened a new log
I20260807 01:45:48.893628 25927 ts_tablet_manager.cc:1403] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de: Time spent bootstrapping tablet: real 0.064s user 0.032s sys 0.012s
I20260807 01:45:48.892326 25952 raft_consensus.cc:515] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.892694 25923 raft_consensus.cc:385] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:48.897401 25923 raft_consensus.cc:740] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b147ae01363d411980d3a99e72915a00, State: Initialized, Role: FOLLOWER
I20260807 01:45:48.898552 25923 consensus_queue.cc:260] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.898667 25927 raft_consensus.cc:359] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.899492 25927 raft_consensus.cc:385] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:48.899848 25927 raft_consensus.cc:740] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f9c53697dc142e4af11d5f5e0e167de, State: Initialized, Role: FOLLOWER
I20260807 01:45:48.899682 25922 consensus_queue.cc:260] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.900903 25927 consensus_queue.cc:260] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.905757 25931 raft_consensus.cc:493] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:48.906296 25931 raft_consensus.cc:515] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.907783 25623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e20a485954424c8b721f0ce6216da2" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
I20260807 01:45:48.908360 25931 leader_election.cc:290] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:48.908471 25623 raft_consensus.cc:3060] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:48.903856 25923 ts_tablet_manager.cc:1434] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00: Time spent starting tablet: real 0.052s user 0.007s sys 0.001s
I20260807 01:45:48.909297 25922 ts_tablet_manager.cc:1434] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8: Time spent starting tablet: real 0.020s user 0.008s sys 0.000s
I20260807 01:45:48.909791 25624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e20a485954424c8b721f0ce6216da2" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" is_pre_election: true
I20260807 01:45:48.910281 25922 tablet_bootstrap.cc:492] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8: Bootstrap starting.
I20260807 01:45:48.911468 25923 tablet_bootstrap.cc:492] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00: Bootstrap starting.
I20260807 01:45:48.914283 25927 ts_tablet_manager.cc:1434] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de: Time spent starting tablet: real 0.020s user 0.005s sys 0.004s
I20260807 01:45:48.916874 25623 raft_consensus.cc:2468] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 1.
I20260807 01:45:48.907907 25952 leader_election.cc:290] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:48.921417 25735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ebd958223c4616b2055f66cfcb5581" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de" is_pre_election: true
W20260807 01:45:48.919978 25798 leader_election.cc:343] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:48.922109 25735 raft_consensus.cc:2468] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 0.
I20260807 01:45:48.923108 25927 tablet_bootstrap.cc:492] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de: Bootstrap starting.
I20260807 01:45:48.925014 25922 tablet_bootstrap.cc:654] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:48.925859 25923 tablet_bootstrap.cc:654] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:48.927131 25573 leader_election.cc:304] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, 8f9c53697dc142e4af11d5f5e0e167de; no voters:
I20260807 01:45:48.928263 25737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06fd48ba8b4b49e2a0f51f1f9b8a67df" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de" is_pre_election: true
I20260807 01:45:48.928931 25798 leader_election.cc:304] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b147ae01363d411980d3a99e72915a00; no voters: 21df8eb83f204bd595e6ecd8f6405ef8, 8f9c53697dc142e4af11d5f5e0e167de
I20260807 01:45:48.928891 25623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ebd958223c4616b2055f66cfcb5581" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" is_pre_election: true
I20260807 01:45:48.932168 25623 raft_consensus.cc:2468] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f9c53697dc142e4af11d5f5e0e167de in term 0.
I20260807 01:45:48.932320 25851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ebd958223c4616b2055f66cfcb5581" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b147ae01363d411980d3a99e72915a00" is_pre_election: true
I20260807 01:45:48.931535 25926 raft_consensus.cc:2749] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 01:45:48.933708 25573 leader_election.cc:343] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:48.933881 25735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a7b6bccb54c42b4895fdfdc6230eea9" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de" is_pre_election: true
I20260807 01:45:48.934016 25682 leader_election.cc:304] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, 8f9c53697dc142e4af11d5f5e0e167de; no voters:
I20260807 01:45:48.934716 25735 raft_consensus.cc:2468] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 0.
I20260807 01:45:48.935357 25933 raft_consensus.cc:2804] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:45:48.935756 25933 raft_consensus.cc:493] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 01:45:48.935925 25684 leader_election.cc:343] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:48.936174 25933 raft_consensus.cc:3060] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:48.938058 25573 leader_election.cc:304] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, 8f9c53697dc142e4af11d5f5e0e167de; no voters:
I20260807 01:45:48.939002 25931 raft_consensus.cc:2804] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:45:48.939375 25931 raft_consensus.cc:493] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:45:48.939726 25931 raft_consensus.cc:3060] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:48.935514 25952 raft_consensus.cc:2804] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:45:48.953459 25927 tablet_bootstrap.cc:654] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:48.955385 25931 raft_consensus.cc:515] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.949986 25952 raft_consensus.cc:493] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:45:48.959005 25952 raft_consensus.cc:3060] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:48.959371 25735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a7b6bccb54c42b4895fdfdc6230eea9" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:45:48.960393 25931 leader_election.cc:290] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 election: Requested vote from peers 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:48.960366 25735 raft_consensus.cc:3060] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:48.963227 25850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a7b6bccb54c42b4895fdfdc6230eea9" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b147ae01363d411980d3a99e72915a00" is_pre_election: true
I20260807 01:45:48.963407 25851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ebd958223c4616b2055f66cfcb5581" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b147ae01363d411980d3a99e72915a00" is_pre_election: true
I20260807 01:45:48.964403 25933 raft_consensus.cc:515] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
W20260807 01:45:48.965183 25575 leader_election.cc:343] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:48.965296 25851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a7b6bccb54c42b4895fdfdc6230eea9" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b147ae01363d411980d3a99e72915a00"
I20260807 01:45:48.966524 25933 leader_election.cc:290] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 1 election: Requested vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:48.966738 25848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06fd48ba8b4b49e2a0f51f1f9b8a67df" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b147ae01363d411980d3a99e72915a00" is_pre_election: true
W20260807 01:45:48.967607 25575 leader_election.cc:343] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:48.967681 25851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ebd958223c4616b2055f66cfcb5581" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b147ae01363d411980d3a99e72915a00"
I20260807 01:45:48.968206 25623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ebd958223c4616b2055f66cfcb5581" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
W20260807 01:45:48.968672 25575 leader_election.cc:343] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:48.969054 25922 tablet_bootstrap.cc:492] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8: No bootstrap required, opened a new log
W20260807 01:45:48.969148 25684 leader_election.cc:343] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:48.969543 25922 ts_tablet_manager.cc:1403] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8: Time spent bootstrapping tablet: real 0.060s user 0.018s sys 0.004s
W20260807 01:45:48.969774 25575 leader_election.cc:343] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:48.970413 25575 leader_election.cc:304] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8; no voters: 8f9c53697dc142e4af11d5f5e0e167de, b147ae01363d411980d3a99e72915a00
I20260807 01:45:48.971796 25931 raft_consensus.cc:2749] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:45:48.973429 25735 raft_consensus.cc:2468] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 1.
I20260807 01:45:48.974023 25922 raft_consensus.cc:359] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.974687 25573 leader_election.cc:304] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, 8f9c53697dc142e4af11d5f5e0e167de; no voters: b147ae01363d411980d3a99e72915a00
I20260807 01:45:48.975157 25922 raft_consensus.cc:385] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:48.975500 25922 raft_consensus.cc:740] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Initialized, Role: FOLLOWER
I20260807 01:45:48.976539 25931 raft_consensus.cc:2804] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:45:48.976402 25922 consensus_queue.cc:260] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.981405 25923 tablet_bootstrap.cc:492] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00: No bootstrap required, opened a new log
I20260807 01:45:48.981998 25923 ts_tablet_manager.cc:1403] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00: Time spent bootstrapping tablet: real 0.071s user 0.003s sys 0.022s
I20260807 01:45:48.985371 25923 raft_consensus.cc:359] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.988938 25923 raft_consensus.cc:385] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:48.989347 25923 raft_consensus.cc:740] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b147ae01363d411980d3a99e72915a00, State: Initialized, Role: FOLLOWER
I20260807 01:45:48.990500 25923 consensus_queue.cc:260] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:48.993026 25923 ts_tablet_manager.cc:1434] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260807 01:45:48.994221 25923 tablet_bootstrap.cc:492] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00: Bootstrap starting.
I20260807 01:45:48.999822 25952 raft_consensus.cc:515] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.000792 25931 raft_consensus.cc:697] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 LEADER]: Becoming Leader. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Running, Role: LEADER
I20260807 01:45:49.001166 25623 raft_consensus.cc:2393] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f9c53697dc142e4af11d5f5e0e167de in current term 1: Already voted for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in this term.
I20260807 01:45:49.001999 25922 ts_tablet_manager.cc:1434] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8: Time spent starting tablet: real 0.032s user 0.012s sys 0.018s
I20260807 01:45:49.001919 25931 consensus_queue.cc:237] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.003064 25682 leader_election.cc:304] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f9c53697dc142e4af11d5f5e0e167de; no voters: 21df8eb83f204bd595e6ecd8f6405ef8, b147ae01363d411980d3a99e72915a00
I20260807 01:45:49.004279 25952 leader_election.cc:290] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 election: Requested vote from peers 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:49.004338 25933 raft_consensus.cc:2749] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 01:45:49.006070 25735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ebd958223c4616b2055f66cfcb5581" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:45:49.006544 25851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ebd958223c4616b2055f66cfcb5581" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b147ae01363d411980d3a99e72915a00"
I20260807 01:45:49.007022 25735 raft_consensus.cc:2393] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 21df8eb83f204bd595e6ecd8f6405ef8 in current term 1: Already voted for candidate 8f9c53697dc142e4af11d5f5e0e167de in this term.
W20260807 01:45:49.008492 25575 leader_election.cc:343] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:49.008762 25922 tablet_bootstrap.cc:492] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8: Bootstrap starting.
I20260807 01:45:49.009450 25923 tablet_bootstrap.cc:654] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:49.009622 25573 leader_election.cc:304] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8; no voters: 8f9c53697dc142e4af11d5f5e0e167de, b147ae01363d411980d3a99e72915a00
I20260807 01:45:49.011103 25952 raft_consensus.cc:2749] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 01:45:49.015600 25927 tablet_bootstrap.cc:492] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de: No bootstrap required, opened a new log
I20260807 01:45:49.016244 25927 ts_tablet_manager.cc:1403] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de: Time spent bootstrapping tablet: real 0.093s user 0.010s sys 0.025s
I20260807 01:45:49.017830 25933 raft_consensus.cc:493] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:49.018474 25933 raft_consensus.cc:515] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.019832 25927 raft_consensus.cc:359] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.020663 25927 raft_consensus.cc:385] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:49.021013 25927 raft_consensus.cc:740] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f9c53697dc142e4af11d5f5e0e167de, State: Initialized, Role: FOLLOWER
I20260807 01:45:49.021875 25927 consensus_queue.cc:260] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.022562 25926 raft_consensus.cc:493] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:49.023365 25922 tablet_bootstrap.cc:654] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:49.023782 25851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fde466361a8d40a48394033c480ecf50" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b147ae01363d411980d3a99e72915a00" is_pre_election: true
I20260807 01:45:49.024068 25926 raft_consensus.cc:515] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.027464 25623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fde466361a8d40a48394033c480ecf50" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" is_pre_election: true
I20260807 01:45:49.024235 25933 leader_election.cc:290] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:49.028265 25623 raft_consensus.cc:2468] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f9c53697dc142e4af11d5f5e0e167de in term 0.
I20260807 01:45:49.028151 25926 leader_election.cc:290] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387)
W20260807 01:45:49.029534 25684 leader_election.cc:343] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:49.030792 25735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7a55cd1670e40ec94e4269dc15517bb" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de" is_pre_election: true
I20260807 01:45:49.031395 25735 raft_consensus.cc:2468] T a7a55cd1670e40ec94e4269dc15517bb P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 0.
I20260807 01:45:49.032687 25799 leader_election.cc:304] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f9c53697dc142e4af11d5f5e0e167de, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:45:49.033576 25926 raft_consensus.cc:2804] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:45:49.033603 25682 leader_election.cc:304] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, 8f9c53697dc142e4af11d5f5e0e167de; no voters: b147ae01363d411980d3a99e72915a00
I20260807 01:45:49.034078 25926 raft_consensus.cc:493] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:45:49.033810 25623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7a55cd1670e40ec94e4269dc15517bb" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" is_pre_election: true
I20260807 01:45:49.034569 25926 raft_consensus.cc:3060] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:49.034767 25623 raft_consensus.cc:2468] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 0.
I20260807 01:45:49.036305 25933 raft_consensus.cc:2804] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:45:49.036787 25933 raft_consensus.cc:493] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:45:49.037166 25933 raft_consensus.cc:3060] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:49.044643 25926 raft_consensus.cc:515] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.047467 25933 raft_consensus.cc:515] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.048089 25623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7a55cd1670e40ec94e4269dc15517bb" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
I20260807 01:45:49.048756 25623 raft_consensus.cc:3060] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:49.050788 25476 catalog_manager.cc:5697] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1). New cstate: current_term: 1 leader_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:49.052933 25851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fde466361a8d40a48394033c480ecf50" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b147ae01363d411980d3a99e72915a00"
I20260807 01:45:49.053294 25923 tablet_bootstrap.cc:492] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00: No bootstrap required, opened a new log
I20260807 01:45:49.053232 25926 leader_election.cc:290] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 election: Requested vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387)
I20260807 01:45:49.053923 25923 ts_tablet_manager.cc:1403] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00: Time spent bootstrapping tablet: real 0.060s user 0.027s sys 0.019s
I20260807 01:45:49.052953 25735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7a55cd1670e40ec94e4269dc15517bb" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:45:49.054781 25735 raft_consensus.cc:3060] T a7a55cd1670e40ec94e4269dc15517bb P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:49.060878 25923 raft_consensus.cc:359] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.061610 25923 raft_consensus.cc:385] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:49.061884 25923 raft_consensus.cc:740] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b147ae01363d411980d3a99e72915a00, State: Initialized, Role: FOLLOWER
I20260807 01:45:49.063043 25923 consensus_queue.cc:260] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.050817 25624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fde466361a8d40a48394033c480ecf50" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
W20260807 01:45:49.062920 25684 leader_election.cc:343] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:49.064929 25735 raft_consensus.cc:2468] T a7a55cd1670e40ec94e4269dc15517bb P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 1.
I20260807 01:45:49.064441 25624 raft_consensus.cc:3060] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:49.066951 25923 ts_tablet_manager.cc:1434] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20260807 01:45:49.063921 25933 leader_election.cc:290] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 1 election: Requested vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:49.067535 25799 leader_election.cc:304] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f9c53697dc142e4af11d5f5e0e167de, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:45:49.072309 25623 raft_consensus.cc:2468] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 1.
I20260807 01:45:49.074337 25923 tablet_bootstrap.cc:492] T a8ebd958223c4616b2055f66cfcb5581 P b147ae01363d411980d3a99e72915a00: Bootstrap starting.
I20260807 01:45:49.080807 25923 tablet_bootstrap.cc:654] T a8ebd958223c4616b2055f66cfcb5581 P b147ae01363d411980d3a99e72915a00: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:49.081310 25624 raft_consensus.cc:2468] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f9c53697dc142e4af11d5f5e0e167de in term 1.
I20260807 01:45:49.081508 25926 raft_consensus.cc:2804] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:45:49.081933 25922 tablet_bootstrap.cc:492] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8: No bootstrap required, opened a new log
I20260807 01:45:49.084306 25922 ts_tablet_manager.cc:1403] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8: Time spent bootstrapping tablet: real 0.076s user 0.013s sys 0.005s
I20260807 01:45:49.084823 25682 leader_election.cc:304] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, 8f9c53697dc142e4af11d5f5e0e167de; no voters: b147ae01363d411980d3a99e72915a00
I20260807 01:45:49.086177 25933 raft_consensus.cc:2804] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:45:49.086642 25933 raft_consensus.cc:697] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 LEADER]: Becoming Leader. State: Replica: 8f9c53697dc142e4af11d5f5e0e167de, State: Running, Role: LEADER
I20260807 01:45:49.086952 25922 raft_consensus.cc:359] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.087545 25933 consensus_queue.cc:237] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.088089 25926 raft_consensus.cc:697] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 1 LEADER]: Becoming Leader. State: Replica: b147ae01363d411980d3a99e72915a00, State: Running, Role: LEADER
I20260807 01:45:49.088923 25926 consensus_queue.cc:237] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.090349 25922 raft_consensus.cc:385] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:49.090647 25922 raft_consensus.cc:740] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Initialized, Role: FOLLOWER
I20260807 01:45:49.092767 25922 consensus_queue.cc:260] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.095448 25922 ts_tablet_manager.cc:1434] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20260807 01:45:49.097920 25927 ts_tablet_manager.cc:1434] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de: Time spent starting tablet: real 0.081s user 0.015s sys 0.016s
I20260807 01:45:49.099196 25927 tablet_bootstrap.cc:492] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de: Bootstrap starting.
I20260807 01:45:49.099793 25922 tablet_bootstrap.cc:492] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8: Bootstrap starting.
I20260807 01:45:49.107203 25980 raft_consensus.cc:493] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:49.107892 25980 raft_consensus.cc:515] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.110792 25922 tablet_bootstrap.cc:654] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:49.116291 25947 raft_consensus.cc:493] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:49.116802 25947 raft_consensus.cc:515] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.117537 25980 leader_election.cc:290] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:49.121773 25947 leader_election.cc:290] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387)
I20260807 01:45:49.122507 25931 raft_consensus.cc:493] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:49.120462 25851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff0dbafbd3f457ba6c59d0e8bf16705" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b147ae01363d411980d3a99e72915a00" is_pre_election: true
I20260807 01:45:49.127540 25735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff0dbafbd3f457ba6c59d0e8bf16705" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de" is_pre_election: true
I20260807 01:45:49.128257 25735 raft_consensus.cc:2468] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 0.
I20260807 01:45:49.128646 25927 tablet_bootstrap.cc:654] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:49.125005 25931 raft_consensus.cc:515] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.129664 25624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff0dbafbd3f457ba6c59d0e8bf16705" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" is_pre_election: true
I20260807 01:45:49.131687 25624 raft_consensus.cc:2468] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f9c53697dc142e4af11d5f5e0e167de in term 0.
I20260807 01:45:49.132015 25799 leader_election.cc:304] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f9c53697dc142e4af11d5f5e0e167de, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:45:49.132956 25923 tablet_bootstrap.cc:492] T a8ebd958223c4616b2055f66cfcb5581 P b147ae01363d411980d3a99e72915a00: No bootstrap required, opened a new log
I20260807 01:45:49.133473 25923 ts_tablet_manager.cc:1403] T a8ebd958223c4616b2055f66cfcb5581 P b147ae01363d411980d3a99e72915a00: Time spent bootstrapping tablet: real 0.059s user 0.014s sys 0.006s
I20260807 01:45:49.129611 25623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff0dbafbd3f457ba6c59d0e8bf16705" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" is_pre_election: true
I20260807 01:45:49.134547 25682 leader_election.cc:304] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, 8f9c53697dc142e4af11d5f5e0e167de; no voters:
I20260807 01:45:49.134618 25848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff0dbafbd3f457ba6c59d0e8bf16705" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b147ae01363d411980d3a99e72915a00" is_pre_election: true
I20260807 01:45:49.132862 25931 leader_election.cc:290] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:49.136252 25623 raft_consensus.cc:2468] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 0.
I20260807 01:45:49.136860 25735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff0dbafbd3f457ba6c59d0e8bf16705" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de" is_pre_election: true
I20260807 01:45:49.137550 25735 raft_consensus.cc:2468] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 0.
I20260807 01:45:49.138849 25573 leader_election.cc:304] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, 8f9c53697dc142e4af11d5f5e0e167de; no voters: b147ae01363d411980d3a99e72915a00
I20260807 01:45:49.139138 25933 raft_consensus.cc:2804] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:45:49.136241 25923 raft_consensus.cc:359] T a8ebd958223c4616b2055f66cfcb5581 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.140118 25923 raft_consensus.cc:385] T a8ebd958223c4616b2055f66cfcb5581 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:49.141517 25923 raft_consensus.cc:740] T a8ebd958223c4616b2055f66cfcb5581 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b147ae01363d411980d3a99e72915a00, State: Initialized, Role: FOLLOWER
I20260807 01:45:49.142405 25923 consensus_queue.cc:260] T a8ebd958223c4616b2055f66cfcb5581 P b147ae01363d411980d3a99e72915a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.139905 25933 raft_consensus.cc:493] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:45:49.141144 25931 raft_consensus.cc:2804] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:45:49.144455 25931 raft_consensus.cc:493] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:45:49.125449 25851 raft_consensus.cc:2468] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f9c53697dc142e4af11d5f5e0e167de in term 0.
I20260807 01:45:49.145989 25931 raft_consensus.cc:3060] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:49.146700 25926 raft_consensus.cc:2804] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:45:49.146227 25476 catalog_manager.cc:5697] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2). New cstate: current_term: 1 leader_uuid: "8f9c53697dc142e4af11d5f5e0e167de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:49.149103 25926 raft_consensus.cc:493] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:45:49.149564 25926 raft_consensus.cc:3060] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:49.153038 25931 raft_consensus.cc:515] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.156297 25931 leader_election.cc:290] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 election: Requested vote from peers 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:49.156581 25851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff0dbafbd3f457ba6c59d0e8bf16705" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b147ae01363d411980d3a99e72915a00"
I20260807 01:45:49.142335 25475 catalog_manager.cc:5697] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 reported cstate change: term changed from 0 to 1, leader changed from <none> to b147ae01363d411980d3a99e72915a00 (127.24.4.3). New cstate: current_term: 1 leader_uuid: "b147ae01363d411980d3a99e72915a00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: HEALTHY } } }
I20260807 01:45:49.160085 25735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff0dbafbd3f457ba6c59d0e8bf16705" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:45:49.161043 25931 raft_consensus.cc:493] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:49.161567 25931 raft_consensus.cc:515] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.144840 25933 raft_consensus.cc:3060] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:49.164697 25737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31f1ba5ca06f461a8b3001ca5c4c48a3" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de" is_pre_election: true
I20260807 01:45:49.166291 25848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31f1ba5ca06f461a8b3001ca5c4c48a3" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b147ae01363d411980d3a99e72915a00" is_pre_election: true
W20260807 01:45:49.166777 25573 leader_election.cc:343] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:49.166955 25848 raft_consensus.cc:2468] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 0.
I20260807 01:45:49.169231 25575 leader_election.cc:304] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, b147ae01363d411980d3a99e72915a00; no voters: 8f9c53697dc142e4af11d5f5e0e167de
I20260807 01:45:49.170306 25975 raft_consensus.cc:2804] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:45:49.170688 25975 raft_consensus.cc:493] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:45:49.171001 25975 raft_consensus.cc:3060] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:49.173151 25923 ts_tablet_manager.cc:1434] T a8ebd958223c4616b2055f66cfcb5581 P b147ae01363d411980d3a99e72915a00: Time spent starting tablet: real 0.039s user 0.007s sys 0.000s
I20260807 01:45:49.174461 25923 tablet_bootstrap.cc:492] T fde466361a8d40a48394033c480ecf50 P b147ae01363d411980d3a99e72915a00: Bootstrap starting.
I20260807 01:45:49.177551 25926 raft_consensus.cc:515] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.179934 25851 raft_consensus.cc:2393] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 21df8eb83f204bd595e6ecd8f6405ef8 in current term 1: Already voted for candidate b147ae01363d411980d3a99e72915a00 in this term.
I20260807 01:45:49.181869 25737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff0dbafbd3f457ba6c59d0e8bf16705" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:45:49.183812 25931 leader_election.cc:290] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:49.185501 25623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff0dbafbd3f457ba6c59d0e8bf16705" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
I20260807 01:45:49.184382 25926 leader_election.cc:290] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 election: Requested vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387)
I20260807 01:45:49.186314 25623 raft_consensus.cc:2393] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b147ae01363d411980d3a99e72915a00 in current term 1: Already voted for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in this term.
I20260807 01:45:49.186537 25922 tablet_bootstrap.cc:492] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8: No bootstrap required, opened a new log
I20260807 01:45:49.187052 25922 ts_tablet_manager.cc:1403] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8: Time spent bootstrapping tablet: real 0.087s user 0.010s sys 0.001s
I20260807 01:45:49.187655 25798 leader_election.cc:304] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b147ae01363d411980d3a99e72915a00; no voters: 21df8eb83f204bd595e6ecd8f6405ef8, 8f9c53697dc142e4af11d5f5e0e167de
I20260807 01:45:49.190413 25926 raft_consensus.cc:2749] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 01:45:49.192314 25922 raft_consensus.cc:359] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.182238 25975 raft_consensus.cc:515] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.193101 25922 raft_consensus.cc:385] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:49.193447 25922 raft_consensus.cc:740] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Initialized, Role: FOLLOWER
I20260807 01:45:49.194211 25922 consensus_queue.cc:260] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.191220 25933 raft_consensus.cc:515] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.195493 25927 tablet_bootstrap.cc:492] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de: No bootstrap required, opened a new log
I20260807 01:45:49.195978 25735 raft_consensus.cc:2393] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 21df8eb83f204bd595e6ecd8f6405ef8 in current term 1: Already voted for candidate 8f9c53697dc142e4af11d5f5e0e167de in this term.
I20260807 01:45:49.196063 25927 ts_tablet_manager.cc:1403] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de: Time spent bootstrapping tablet: real 0.097s user 0.023s sys 0.040s
I20260807 01:45:49.197394 25737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31f1ba5ca06f461a8b3001ca5c4c48a3" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:45:49.199226 25573 leader_election.cc:304] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8; no voters: 8f9c53697dc142e4af11d5f5e0e167de, b147ae01363d411980d3a99e72915a00
I20260807 01:45:49.199345 25927 raft_consensus.cc:359] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.199731 25926 raft_consensus.cc:493] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:49.200307 25931 raft_consensus.cc:2749] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 01:45:49.200755 25926 raft_consensus.cc:515] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.202822 25923 tablet_bootstrap.cc:654] T fde466361a8d40a48394033c480ecf50 P b147ae01363d411980d3a99e72915a00: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:49.203213 25926 leader_election.cc:290] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387)
I20260807 01:45:49.200225 25927 raft_consensus.cc:385] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:49.203989 25927 raft_consensus.cc:740] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f9c53697dc142e4af11d5f5e0e167de, State: Initialized, Role: FOLLOWER
I20260807 01:45:49.204416 25922 ts_tablet_manager.cc:1434] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8: Time spent starting tablet: real 0.017s user 0.008s sys 0.001s
I20260807 01:45:49.204535 25933 leader_election.cc:290] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 1 election: Requested vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:49.205456 25922 tablet_bootstrap.cc:492] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8: Bootstrap starting.
I20260807 01:45:49.204901 25927 consensus_queue.cc:260] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.207358 25927 ts_tablet_manager.cc:1434] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260807 01:45:49.208559 25927 tablet_bootstrap.cc:492] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de: Bootstrap starting.
I20260807 01:45:49.205299 25623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff0dbafbd3f457ba6c59d0e8bf16705" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
I20260807 01:45:49.209492 25623 raft_consensus.cc:2393] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f9c53697dc142e4af11d5f5e0e167de in current term 1: Already voted for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in this term.
I20260807 01:45:49.213603 25922 tablet_bootstrap.cc:654] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:49.214773 25927 tablet_bootstrap.cc:654] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:49.216315 25737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39269a566fe848f481ab0b1a50d48220" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de" is_pre_election: true
I20260807 01:45:49.217060 25737 raft_consensus.cc:2468] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 0.
I20260807 01:45:49.218609 25799 leader_election.cc:304] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f9c53697dc142e4af11d5f5e0e167de, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:45:49.219506 25926 raft_consensus.cc:2804] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:45:49.219905 25926 raft_consensus.cc:493] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:45:49.220232 25926 raft_consensus.cc:3060] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:49.220852 25923 tablet_bootstrap.cc:492] T fde466361a8d40a48394033c480ecf50 P b147ae01363d411980d3a99e72915a00: No bootstrap required, opened a new log
I20260807 01:45:49.220685 25851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff0dbafbd3f457ba6c59d0e8bf16705" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b147ae01363d411980d3a99e72915a00"
I20260807 01:45:49.221320 25923 ts_tablet_manager.cc:1403] T fde466361a8d40a48394033c480ecf50 P b147ae01363d411980d3a99e72915a00: Time spent bootstrapping tablet: real 0.047s user 0.013s sys 0.008s
I20260807 01:45:49.221478 25851 raft_consensus.cc:2393] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f9c53697dc142e4af11d5f5e0e167de in current term 1: Already voted for candidate b147ae01363d411980d3a99e72915a00 in this term.
I20260807 01:45:49.222838 25684 leader_election.cc:304] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f9c53697dc142e4af11d5f5e0e167de; no voters: 21df8eb83f204bd595e6ecd8f6405ef8, b147ae01363d411980d3a99e72915a00
I20260807 01:45:49.224054 25933 raft_consensus.cc:2749] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260807 01:45:49.200563 25573 leader_election.cc:343] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:49.226419 25623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39269a566fe848f481ab0b1a50d48220" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" is_pre_election: true
I20260807 01:45:49.227499 25923 raft_consensus.cc:359] T fde466361a8d40a48394033c480ecf50 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
W20260807 01:45:49.227957 25798 leader_election.cc:343] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:45:49.228255 25923 raft_consensus.cc:385] T fde466361a8d40a48394033c480ecf50 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:49.228567 25923 raft_consensus.cc:740] T fde466361a8d40a48394033c480ecf50 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b147ae01363d411980d3a99e72915a00, State: Initialized, Role: FOLLOWER
I20260807 01:45:49.228915 25926 raft_consensus.cc:515] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.229439 25923 consensus_queue.cc:260] T fde466361a8d40a48394033c480ecf50 P b147ae01363d411980d3a99e72915a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.231189 25926 leader_election.cc:290] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 election: Requested vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387)
I20260807 01:45:49.234022 25922 tablet_bootstrap.cc:492] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8: No bootstrap required, opened a new log
I20260807 01:45:49.234591 25922 ts_tablet_manager.cc:1403] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8: Time spent bootstrapping tablet: real 0.029s user 0.012s sys 0.007s
I20260807 01:45:49.235378 25923 ts_tablet_manager.cc:1434] T fde466361a8d40a48394033c480ecf50 P b147ae01363d411980d3a99e72915a00: Time spent starting tablet: real 0.014s user 0.009s sys 0.000s
I20260807 01:45:49.236783 25927 tablet_bootstrap.cc:492] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de: No bootstrap required, opened a new log
I20260807 01:45:49.236865 25737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39269a566fe848f481ab0b1a50d48220" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:45:49.237454 25927 ts_tablet_manager.cc:1403] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de: Time spent bootstrapping tablet: real 0.029s user 0.014s sys 0.000s
I20260807 01:45:49.237236 25922 raft_consensus.cc:359] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.237558 25737 raft_consensus.cc:3060] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:49.241618 25927 raft_consensus.cc:359] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.242645 25927 raft_consensus.cc:385] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:49.243166 25927 raft_consensus.cc:740] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f9c53697dc142e4af11d5f5e0e167de, State: Initialized, Role: FOLLOWER
I20260807 01:45:49.244134 25927 consensus_queue.cc:260] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.245494 25737 raft_consensus.cc:2468] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 1.
I20260807 01:45:49.245848 25933 raft_consensus.cc:493] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:49.246410 25927 ts_tablet_manager.cc:1434] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260807 01:45:49.246660 25933 raft_consensus.cc:515] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.247356 25927 tablet_bootstrap.cc:492] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 8f9c53697dc142e4af11d5f5e0e167de: Bootstrap starting.
I20260807 01:45:49.249156 25623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39269a566fe848f481ab0b1a50d48220" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
I20260807 01:45:49.250105 25799 leader_election.cc:304] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f9c53697dc142e4af11d5f5e0e167de, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:45:49.251068 25926 raft_consensus.cc:2804] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:45:49.251075 25947 raft_consensus.cc:493] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:49.251240 25851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061b319309704115ac6bc2716e08fd4a" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b147ae01363d411980d3a99e72915a00" is_pre_election: true
I20260807 01:45:49.252002 25851 raft_consensus.cc:2468] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f9c53697dc142e4af11d5f5e0e167de in term 0.
I20260807 01:45:49.251794 25947 raft_consensus.cc:515] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.252354 25624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061b319309704115ac6bc2716e08fd4a" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" is_pre_election: true
I20260807 01:45:49.253015 25624 raft_consensus.cc:2468] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f9c53697dc142e4af11d5f5e0e167de in term 0.
I20260807 01:45:49.253818 25947 leader_election.cc:290] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387)
I20260807 01:45:49.254856 25682 leader_election.cc:304] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, 8f9c53697dc142e4af11d5f5e0e167de; no voters:
I20260807 01:45:49.255501 25933 leader_election.cc:290] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:49.255774 25737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a7b6bccb54c42b4895fdfdc6230eea9" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de" is_pre_election: true
I20260807 01:45:49.256671 25737 raft_consensus.cc:2393] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b147ae01363d411980d3a99e72915a00 in current term 1: Already voted for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in this term.
I20260807 01:45:49.256582 25980 raft_consensus.cc:2804] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:45:49.257786 25980 raft_consensus.cc:493] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:45:49.258143 25980 raft_consensus.cc:3060] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:49.261759 25624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a7b6bccb54c42b4895fdfdc6230eea9" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" is_pre_election: true
I20260807 01:45:49.263471 25798 leader_election.cc:304] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b147ae01363d411980d3a99e72915a00; no voters: 21df8eb83f204bd595e6ecd8f6405ef8, 8f9c53697dc142e4af11d5f5e0e167de
I20260807 01:45:49.264348 25947 raft_consensus.cc:3060] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:49.263475 25927 tablet_bootstrap.cc:654] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 8f9c53697dc142e4af11d5f5e0e167de: Neither blocks nor log segments found. Creating new log.
I20260807 01:45:49.264689 25851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31f1ba5ca06f461a8b3001ca5c4c48a3" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b147ae01363d411980d3a99e72915a00"
I20260807 01:45:49.265550 25851 raft_consensus.cc:3060] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:49.245887 25922 raft_consensus.cc:385] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:49.268417 25922 raft_consensus.cc:740] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Initialized, Role: FOLLOWER
I20260807 01:45:49.269299 25975 leader_election.cc:290] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 election: Requested vote from peers 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:49.269520 25922 consensus_queue.cc:260] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.272663 25623 raft_consensus.cc:3060] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:49.277737 25947 raft_consensus.cc:2749] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:45:49.282663 25926 raft_consensus.cc:697] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 1 LEADER]: Becoming Leader. State: Replica: b147ae01363d411980d3a99e72915a00, State: Running, Role: LEADER
I20260807 01:45:49.284055 25623 raft_consensus.cc:2468] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 1.
I20260807 01:45:49.284775 25926 consensus_queue.cc:237] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.285600 25927 tablet_bootstrap.cc:492] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 8f9c53697dc142e4af11d5f5e0e167de: No bootstrap required, opened a new log
I20260807 01:45:49.286065 25927 ts_tablet_manager.cc:1403] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 8f9c53697dc142e4af11d5f5e0e167de: Time spent bootstrapping tablet: real 0.039s user 0.007s sys 0.004s
I20260807 01:45:49.286995 25980 raft_consensus.cc:515] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.288038 25947 consensus_queue.cc:1048] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:45:49.289618 25927 raft_consensus.cc:359] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.290441 25927 raft_consensus.cc:385] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:45:49.290726 25927 raft_consensus.cc:740] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f9c53697dc142e4af11d5f5e0e167de, State: Initialized, Role: FOLLOWER
I20260807 01:45:49.291566 25927 consensus_queue.cc:260] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 8f9c53697dc142e4af11d5f5e0e167de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.293787 25851 raft_consensus.cc:2468] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 1.
I20260807 01:45:49.294992 25623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061b319309704115ac6bc2716e08fd4a" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
I20260807 01:45:49.295675 25623 raft_consensus.cc:3060] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:49.295531 25575 leader_election.cc:304] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, b147ae01363d411980d3a99e72915a00; no voters: 8f9c53697dc142e4af11d5f5e0e167de
I20260807 01:45:49.296700 25927 ts_tablet_manager.cc:1434] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 8f9c53697dc142e4af11d5f5e0e167de: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260807 01:45:49.301465 25975 raft_consensus.cc:2804] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:45:49.301993 25975 raft_consensus.cc:697] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 LEADER]: Becoming Leader. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Running, Role: LEADER
I20260807 01:45:49.302995 25975 consensus_queue.cc:237] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.310972 25922 ts_tablet_manager.cc:1434] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8: Time spent starting tablet: real 0.076s user 0.002s sys 0.008s
I20260807 01:45:49.314267 25980 leader_election.cc:290] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 1 election: Requested vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:49.317699 25476 catalog_manager.cc:5697] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 reported cstate change: term changed from 0 to 1, leader changed from <none> to b147ae01363d411980d3a99e72915a00 (127.24.4.3). New cstate: current_term: 1 leader_uuid: "b147ae01363d411980d3a99e72915a00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: HEALTHY } } }
I20260807 01:45:49.321800 25851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061b319309704115ac6bc2716e08fd4a" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b147ae01363d411980d3a99e72915a00"
I20260807 01:45:49.322659 25851 raft_consensus.cc:3060] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:49.334046 26003 consensus_queue.cc:1048] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260807 01:45:49.336153 25623 raft_consensus.cc:2468] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f9c53697dc142e4af11d5f5e0e167de in term 1.
I20260807 01:45:49.345180 25475 catalog_manager.cc:5697] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1). New cstate: current_term: 1 leader_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:49.342756 25851 raft_consensus.cc:2468] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f9c53697dc142e4af11d5f5e0e167de in term 1.
I20260807 01:45:49.347332 25682 leader_election.cc:304] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, 8f9c53697dc142e4af11d5f5e0e167de; no voters:
I20260807 01:45:49.348217 25980 raft_consensus.cc:2804] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:45:49.348810 25980 raft_consensus.cc:697] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 1 LEADER]: Becoming Leader. State: Replica: 8f9c53697dc142e4af11d5f5e0e167de, State: Running, Role: LEADER
I20260807 01:45:49.349804 25980 consensus_queue.cc:237] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.374639 25475 catalog_manager.cc:5697] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2). New cstate: current_term: 1 leader_uuid: "8f9c53697dc142e4af11d5f5e0e167de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:49.459904 25975 consensus_queue.cc:1048] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:45:49.471153 25931 consensus_queue.cc:1048] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:45:49.493440 25947 raft_consensus.cc:493] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:49.493917 25947 raft_consensus.cc:515] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.495601 25947 leader_election.cc:290] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387)
I20260807 01:45:49.496775 25737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06fd48ba8b4b49e2a0f51f1f9b8a67df" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de" is_pre_election: true
I20260807 01:45:49.496775 25624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06fd48ba8b4b49e2a0f51f1f9b8a67df" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" is_pre_election: true
I20260807 01:45:49.497642 25624 raft_consensus.cc:2468] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 0.
I20260807 01:45:49.497610 25737 raft_consensus.cc:2468] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 0.
I20260807 01:45:49.498999 25799 leader_election.cc:304] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f9c53697dc142e4af11d5f5e0e167de, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:45:49.499682 25947 raft_consensus.cc:2804] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:45:49.500022 25947 raft_consensus.cc:493] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:45:49.500253 25947 raft_consensus.cc:3060] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:49.504956 25947 raft_consensus.cc:515] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.506891 25947 leader_election.cc:290] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 election: Requested vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387)
I20260807 01:45:49.507983 25737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06fd48ba8b4b49e2a0f51f1f9b8a67df" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:45:49.507983 25623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06fd48ba8b4b49e2a0f51f1f9b8a67df" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
I20260807 01:45:49.508733 25737 raft_consensus.cc:3060] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:49.508857 25623 raft_consensus.cc:3060] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:49.508773 25947 consensus_queue.cc:1048] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:45:49.515311 25737 raft_consensus.cc:2468] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 1.
I20260807 01:45:49.516512 25799 leader_election.cc:304] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f9c53697dc142e4af11d5f5e0e167de, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:45:49.517607 25926 raft_consensus.cc:2804] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:45:49.518594 25623 raft_consensus.cc:2468] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 1.
I20260807 01:45:49.522871 26003 raft_consensus.cc:493] T a8ebd958223c4616b2055f66cfcb5581 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:49.523788 26003 raft_consensus.cc:515] T a8ebd958223c4616b2055f66cfcb5581 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.525220 25947 consensus_queue.cc:1048] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:45:49.525939 26003 leader_election.cc:290] T a8ebd958223c4616b2055f66cfcb5581 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387)
I20260807 01:45:49.527418 25622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ebd958223c4616b2055f66cfcb5581" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" is_pre_election: true
I20260807 01:45:49.527640 25926 raft_consensus.cc:697] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 1 LEADER]: Becoming Leader. State: Replica: b147ae01363d411980d3a99e72915a00, State: Running, Role: LEADER
I20260807 01:45:49.528677 25622 raft_consensus.cc:2393] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b147ae01363d411980d3a99e72915a00 in current term 1: Already voted for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in this term.
I20260807 01:45:49.528849 25737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ebd958223c4616b2055f66cfcb5581" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de" is_pre_election: true
I20260807 01:45:49.528976 25926 consensus_queue.cc:237] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.529628 25737 raft_consensus.cc:2393] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b147ae01363d411980d3a99e72915a00 in current term 1: Already voted for candidate 8f9c53697dc142e4af11d5f5e0e167de in this term.
I20260807 01:45:49.532830 25799 leader_election.cc:304] T a8ebd958223c4616b2055f66cfcb5581 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b147ae01363d411980d3a99e72915a00; no voters: 21df8eb83f204bd595e6ecd8f6405ef8, 8f9c53697dc142e4af11d5f5e0e167de
I20260807 01:45:49.535209 26026 raft_consensus.cc:3060] T a8ebd958223c4616b2055f66cfcb5581 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:49.541918 26026 raft_consensus.cc:2749] T a8ebd958223c4616b2055f66cfcb5581 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:45:49.548357 25475 catalog_manager.cc:5697] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 reported cstate change: term changed from 0 to 1, leader changed from <none> to b147ae01363d411980d3a99e72915a00 (127.24.4.3). New cstate: current_term: 1 leader_uuid: "b147ae01363d411980d3a99e72915a00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: HEALTHY } } }
I20260807 01:45:49.581367 25926 raft_consensus.cc:493] T fde466361a8d40a48394033c480ecf50 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:49.581979 25926 raft_consensus.cc:515] T fde466361a8d40a48394033c480ecf50 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.583666 25926 leader_election.cc:290] T fde466361a8d40a48394033c480ecf50 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387)
I20260807 01:45:49.584563 25622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fde466361a8d40a48394033c480ecf50" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" is_pre_election: true
I20260807 01:45:49.584685 25737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fde466361a8d40a48394033c480ecf50" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de" is_pre_election: true
I20260807 01:45:49.585239 25622 raft_consensus.cc:2393] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b147ae01363d411980d3a99e72915a00 in current term 1: Already voted for candidate 8f9c53697dc142e4af11d5f5e0e167de in this term.
I20260807 01:45:49.586216 25798 leader_election.cc:304] T fde466361a8d40a48394033c480ecf50 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b147ae01363d411980d3a99e72915a00; no voters: 21df8eb83f204bd595e6ecd8f6405ef8, 8f9c53697dc142e4af11d5f5e0e167de
I20260807 01:45:49.586920 25926 raft_consensus.cc:3060] T fde466361a8d40a48394033c480ecf50 P b147ae01363d411980d3a99e72915a00 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:49.592254 25926 raft_consensus.cc:2749] T fde466361a8d40a48394033c480ecf50 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:45:49.598914 25980 raft_consensus.cc:493] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:49.599431 25980 raft_consensus.cc:515] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:49.601069 25980 leader_election.cc:290] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:49.602074 25622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31f1ba5ca06f461a8b3001ca5c4c48a3" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" is_pre_election: true
I20260807 01:45:49.602236 25851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31f1ba5ca06f461a8b3001ca5c4c48a3" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b147ae01363d411980d3a99e72915a00" is_pre_election: true
I20260807 01:45:49.603130 25851 raft_consensus.cc:2393] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f9c53697dc142e4af11d5f5e0e167de in current term 1: Already voted for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in this term.
I20260807 01:45:49.604115 25684 leader_election.cc:304] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f9c53697dc142e4af11d5f5e0e167de; no voters: 21df8eb83f204bd595e6ecd8f6405ef8, b147ae01363d411980d3a99e72915a00
I20260807 01:45:49.604759 25980 raft_consensus.cc:3060] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 8f9c53697dc142e4af11d5f5e0e167de [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:45:49.609838 25980 raft_consensus.cc:2749] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:45:49.653388 25980 consensus_queue.cc:1048] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [LEADER]: Connected to new peer: Peer: permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:45:49.658357 25980 consensus_queue.cc:1048] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [LEADER]: Connected to new peer: Peer: permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:45:49.758544 25926 consensus_queue.cc:1048] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:45:49.770895 25931 consensus_queue.cc:1048] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:45:49.773841 26039 consensus_queue.cc:1048] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:45:49.787612 25931 consensus_queue.cc:1048] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 01:45:49.796386 25541 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 01:45:49.800896 26032 consensus_queue.cc:1048] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [LEADER]: Connected to new peer: Peer: permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:45:49.813582 26032 consensus_queue.cc:1048] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [LEADER]: Connected to new peer: Peer: permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:45:49.955966 26039 consensus_queue.cc:1048] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:45:49.978454 25926 consensus_queue.cc:1048] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260807 01:45:50.798543 25541 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 01:45:50.938668 26039 raft_consensus.cc:493] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:50.939242 26039 raft_consensus.cc:515] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:50.941215 26039 leader_election.cc:290] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387)
I20260807 01:45:50.942062 25622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff0dbafbd3f457ba6c59d0e8bf16705" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" is_pre_election: true
I20260807 01:45:50.942270 25737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff0dbafbd3f457ba6c59d0e8bf16705" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de" is_pre_election: true
I20260807 01:45:50.942903 25622 raft_consensus.cc:2468] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 1.
I20260807 01:45:50.942914 25737 raft_consensus.cc:2468] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 1.
I20260807 01:45:50.944404 25798 leader_election.cc:304] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:45:50.945112 26039 raft_consensus.cc:2804] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:45:50.945411 26039 raft_consensus.cc:493] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:45:50.945703 26039 raft_consensus.cc:3060] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:50.950800 26039 raft_consensus.cc:515] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:50.952436 26039 leader_election.cc:290] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 2 election: Requested vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387)
I20260807 01:45:50.953419 25622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff0dbafbd3f457ba6c59d0e8bf16705" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
I20260807 01:45:50.953526 25737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff0dbafbd3f457ba6c59d0e8bf16705" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:45:50.953950 25622 raft_consensus.cc:3060] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:50.954083 25737 raft_consensus.cc:3060] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:50.959231 25737 raft_consensus.cc:2468] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 2.
I20260807 01:45:50.959522 25622 raft_consensus.cc:2468] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 2.
I20260807 01:45:50.960496 25799 leader_election.cc:304] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f9c53697dc142e4af11d5f5e0e167de, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:45:50.961243 26039 raft_consensus.cc:2804] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:45:50.961650 26039 raft_consensus.cc:697] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 2 LEADER]: Becoming Leader. State: Replica: b147ae01363d411980d3a99e72915a00, State: Running, Role: LEADER
I20260807 01:45:50.962311 26039 consensus_queue.cc:237] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:50.971025 25472 catalog_manager.cc:5697] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 reported cstate change: term changed from 0 to 2, leader changed from <none> to b147ae01363d411980d3a99e72915a00 (127.24.4.3). New cstate: current_term: 2 leader_uuid: "b147ae01363d411980d3a99e72915a00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: HEALTHY } } }
I20260807 01:45:51.089540 25974 raft_consensus.cc:493] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:45:51.090019 25974 raft_consensus.cc:515] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:51.092031 25974 leader_election.cc:290] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:51.093391 25851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ebd958223c4616b2055f66cfcb5581" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b147ae01363d411980d3a99e72915a00" is_pre_election: true
I20260807 01:45:51.093142 25737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ebd958223c4616b2055f66cfcb5581" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de" is_pre_election: true
I20260807 01:45:51.093927 25851 raft_consensus.cc:2468] T a8ebd958223c4616b2055f66cfcb5581 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 1.
I20260807 01:45:51.094259 25737 raft_consensus.cc:2468] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 1.
I20260807 01:45:51.095141 25575 leader_election.cc:304] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:45:51.095983 25974 raft_consensus.cc:2804] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:45:51.096293 25974 raft_consensus.cc:493] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:45:51.096516 25974 raft_consensus.cc:3060] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:51.101617 25974 raft_consensus.cc:515] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:51.103500 25974 leader_election.cc:290] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 2 election: Requested vote from peers 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:51.104558 25737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ebd958223c4616b2055f66cfcb5581" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:45:51.104599 25851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ebd958223c4616b2055f66cfcb5581" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b147ae01363d411980d3a99e72915a00"
I20260807 01:45:51.105295 25737 raft_consensus.cc:3060] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:51.105331 25851 raft_consensus.cc:3060] T a8ebd958223c4616b2055f66cfcb5581 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:51.111527 25851 raft_consensus.cc:2468] T a8ebd958223c4616b2055f66cfcb5581 P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 2.
I20260807 01:45:51.111668 25737 raft_consensus.cc:2468] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 2.
I20260807 01:45:51.112735 25575 leader_election.cc:304] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:45:51.113471 25974 raft_consensus.cc:2804] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:45:51.113947 25974 raft_consensus.cc:697] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 LEADER]: Becoming Leader. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Running, Role: LEADER
I20260807 01:45:51.114660 25974 consensus_queue.cc:237] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:51.121944 25472 catalog_manager.cc:5697] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 reported cstate change: term changed from 0 to 2, leader changed from <none> to 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:51.473138 25737 raft_consensus.cc:1275] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Refusing update from remote peer b147ae01363d411980d3a99e72915a00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260807 01:45:51.474594 26039 consensus_queue.cc:1048] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:45:51.486390 25622 raft_consensus.cc:1275] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Refusing update from remote peer b147ae01363d411980d3a99e72915a00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260807 01:45:51.488157 26074 consensus_queue.cc:1048] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:45:51.539647 25851 raft_consensus.cc:1275] T a8ebd958223c4616b2055f66cfcb5581 P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Refusing update from remote peer 21df8eb83f204bd595e6ecd8f6405ef8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260807 01:45:51.541002 26066 consensus_queue.cc:1048] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:45:51.551019 25737 raft_consensus.cc:1275] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Refusing update from remote peer 21df8eb83f204bd595e6ecd8f6405ef8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260807 01:45:51.552670 26066 consensus_queue.cc:1048] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:45:51.799256 25851 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06fd48ba8b4b49e2a0f51f1f9b8a67df"
dest_uuid: "b147ae01363d411980d3a99e72915a00"
mode: GRACEFUL
new_leader_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
from {username='slave'} at 127.0.0.1:36834
I20260807 01:45:51.799860 25851 raft_consensus.cc:606] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 1 LEADER]: Received request to transfer leadership to TS 21df8eb83f204bd595e6ecd8f6405ef8
I20260807 01:45:51.816202 25737 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fde466361a8d40a48394033c480ecf50"
dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
mode: GRACEFUL
new_leader_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
from {username='slave'} at 127.0.0.1:41602
I20260807 01:45:51.816865 25737 raft_consensus.cc:606] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 LEADER]: Received request to transfer leadership to TS 21df8eb83f204bd595e6ecd8f6405ef8
I20260807 01:45:51.827267 25851 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a7a55cd1670e40ec94e4269dc15517bb"
dest_uuid: "b147ae01363d411980d3a99e72915a00"
mode: GRACEFUL
new_leader_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
from {username='slave'} at 127.0.0.1:36834
I20260807 01:45:51.828392 25851 raft_consensus.cc:606] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 1 LEADER]: Received request to transfer leadership to TS 21df8eb83f204bd595e6ecd8f6405ef8
I20260807 01:45:51.838027 25851 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39269a566fe848f481ab0b1a50d48220"
dest_uuid: "b147ae01363d411980d3a99e72915a00"
mode: GRACEFUL
new_leader_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
from {username='slave'} at 127.0.0.1:36834
I20260807 01:45:51.838920 25851 raft_consensus.cc:606] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 1 LEADER]: Received request to transfer leadership to TS 8f9c53697dc142e4af11d5f5e0e167de
I20260807 01:45:51.848927 25851 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75e20a485954424c8b721f0ce6216da2"
dest_uuid: "b147ae01363d411980d3a99e72915a00"
mode: GRACEFUL
new_leader_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
from {username='slave'} at 127.0.0.1:36834
I20260807 01:45:51.849781 25851 raft_consensus.cc:606] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 1 LEADER]: Received request to transfer leadership to TS 8f9c53697dc142e4af11d5f5e0e167de
I20260807 01:45:51.861193 25851 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eff0dbafbd3f457ba6c59d0e8bf16705"
dest_uuid: "b147ae01363d411980d3a99e72915a00"
mode: GRACEFUL
new_leader_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
from {username='slave'} at 127.0.0.1:36834
I20260807 01:45:51.861956 25851 raft_consensus.cc:606] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 2 LEADER]: Received request to transfer leadership to TS 8f9c53697dc142e4af11d5f5e0e167de
I20260807 01:45:51.877360 25622 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31f1ba5ca06f461a8b3001ca5c4c48a3"
dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
mode: GRACEFUL
new_leader_uuid: "b147ae01363d411980d3a99e72915a00"
from {username='slave'} at 127.0.0.1:51132
I20260807 01:45:51.878006 25622 raft_consensus.cc:606] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 LEADER]: Received request to transfer leadership to TS b147ae01363d411980d3a99e72915a00
I20260807 01:45:51.971503 26074 raft_consensus.cc:993] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00: : Instructing follower 8f9c53697dc142e4af11d5f5e0e167de to start an election
I20260807 01:45:51.972005 26074 raft_consensus.cc:1081] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 2 LEADER]: Signalling peer 8f9c53697dc142e4af11d5f5e0e167de to start an election
I20260807 01:45:51.973527 25735 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eff0dbafbd3f457ba6c59d0e8bf16705"
dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
from {username='slave'} at 127.0.0.1:41584
I20260807 01:45:51.974056 25735 raft_consensus.cc:493] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:45:51.974407 25735 raft_consensus.cc:3060] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:45:51.974844 26074 raft_consensus.cc:993] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00: : Instructing follower 21df8eb83f204bd595e6ecd8f6405ef8 to start an election
I20260807 01:45:51.975309 26039 raft_consensus.cc:1081] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 1 LEADER]: Signalling peer 21df8eb83f204bd595e6ecd8f6405ef8 to start an election
I20260807 01:45:51.976752 25622 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06fd48ba8b4b49e2a0f51f1f9b8a67df"
dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
from {username='slave'} at 127.0.0.1:51114
I20260807 01:45:51.977224 25622 raft_consensus.cc:493] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:45:51.977500 25622 raft_consensus.cc:3060] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:51.979585 25735 raft_consensus.cc:515] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:51.981312 25735 leader_election.cc:290] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 3 election: Requested vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:51.982388 25624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff0dbafbd3f457ba6c59d0e8bf16705" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
I20260807 01:45:51.982537 25851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff0dbafbd3f457ba6c59d0e8bf16705" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "b147ae01363d411980d3a99e72915a00"
I20260807 01:45:51.983019 25624 raft_consensus.cc:3060] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:45:51.982774 25622 raft_consensus.cc:515] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:51.983114 25851 raft_consensus.cc:3055] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:45:51.983520 25851 raft_consensus.cc:740] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b147ae01363d411980d3a99e72915a00, State: Running, Role: LEADER
I20260807 01:45:51.984876 25851 consensus_queue.cc:260] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:51.986593 25851 raft_consensus.cc:3060] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:45:51.986958 25848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06fd48ba8b4b49e2a0f51f1f9b8a67df" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b147ae01363d411980d3a99e72915a00"
I20260807 01:45:51.986513 25735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06fd48ba8b4b49e2a0f51f1f9b8a67df" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:45:51.987567 25848 raft_consensus.cc:3055] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:45:51.987636 25735 raft_consensus.cc:3060] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:51.988047 25848 raft_consensus.cc:740] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b147ae01363d411980d3a99e72915a00, State: Running, Role: LEADER
I20260807 01:45:51.989146 25848 consensus_queue.cc:260] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:51.990461 25848 raft_consensus.cc:3060] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:51.991122 25624 raft_consensus.cc:2468] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f9c53697dc142e4af11d5f5e0e167de in term 3.
I20260807 01:45:51.992563 25682 leader_election.cc:304] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, 8f9c53697dc142e4af11d5f5e0e167de; no voters:
I20260807 01:45:51.987061 25622 leader_election.cc:290] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 2 election: Requested vote from peers 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:51.993384 26052 raft_consensus.cc:2804] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:45:51.994967 25851 raft_consensus.cc:2468] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f9c53697dc142e4af11d5f5e0e167de in term 3.
I20260807 01:45:51.995534 25735 raft_consensus.cc:2468] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 2.
I20260807 01:45:51.996541 26052 raft_consensus.cc:697] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 3 LEADER]: Becoming Leader. State: Replica: 8f9c53697dc142e4af11d5f5e0e167de, State: Running, Role: LEADER
I20260807 01:45:51.996987 25573 leader_election.cc:304] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, 8f9c53697dc142e4af11d5f5e0e167de; no voters:
I20260807 01:45:51.997929 26087 raft_consensus.cc:2804] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:45:51.997890 26052 consensus_queue.cc:237] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:51.998543 26087 raft_consensus.cc:697] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 LEADER]: Becoming Leader. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Running, Role: LEADER
I20260807 01:45:51.999785 25848 raft_consensus.cc:2468] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 2.
I20260807 01:45:51.999497 26087 consensus_queue.cc:237] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:52.008841 25472 catalog_manager.cc:5697] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de reported cstate change: term changed from 2 to 3, leader changed from b147ae01363d411980d3a99e72915a00 (127.24.4.3) to 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2). New cstate: current_term: 3 leader_uuid: "8f9c53697dc142e4af11d5f5e0e167de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:52.015934 25471 catalog_manager.cc:5697] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 reported cstate change: term changed from 1 to 2, leader changed from b147ae01363d411980d3a99e72915a00 (127.24.4.3) to 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:52.140002 26089 raft_consensus.cc:993] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de: : Instructing follower 21df8eb83f204bd595e6ecd8f6405ef8 to start an election
I20260807 01:45:52.140456 26052 raft_consensus.cc:1081] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 LEADER]: Signalling peer 21df8eb83f204bd595e6ecd8f6405ef8 to start an election
I20260807 01:45:52.142369 25622 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fde466361a8d40a48394033c480ecf50"
dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
from {username='slave'} at 127.0.0.1:51124
I20260807 01:45:52.142855 25622 raft_consensus.cc:493] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:45:52.143151 25622 raft_consensus.cc:3060] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:52.149281 25622 raft_consensus.cc:515] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:52.151095 25622 leader_election.cc:290] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 2 election: Requested vote from peers 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:52.152176 25848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fde466361a8d40a48394033c480ecf50" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b147ae01363d411980d3a99e72915a00"
I20260807 01:45:52.152350 25735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fde466361a8d40a48394033c480ecf50" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:45:52.152871 25735 raft_consensus.cc:3055] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:45:52.152837 25848 raft_consensus.cc:3060] T fde466361a8d40a48394033c480ecf50 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:52.153319 25735 raft_consensus.cc:740] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f9c53697dc142e4af11d5f5e0e167de, State: Running, Role: LEADER
I20260807 01:45:52.154227 25735 consensus_queue.cc:260] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:52.155524 25735 raft_consensus.cc:3060] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:52.160576 25848 raft_consensus.cc:2468] T fde466361a8d40a48394033c480ecf50 P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 2.
I20260807 01:45:52.162329 25575 leader_election.cc:304] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:45:52.162482 25735 raft_consensus.cc:2468] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 2.
I20260807 01:45:52.163231 26087 raft_consensus.cc:2804] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:45:52.163707 26087 raft_consensus.cc:697] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 LEADER]: Becoming Leader. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Running, Role: LEADER
I20260807 01:45:52.164474 26087 consensus_queue.cc:237] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:52.173552 25471 catalog_manager.cc:5697] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8 reported cstate change: term changed from 1 to 2, leader changed from 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2) to 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:52.252099 26074 raft_consensus.cc:993] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00: : Instructing follower 8f9c53697dc142e4af11d5f5e0e167de to start an election
I20260807 01:45:52.252861 26039 raft_consensus.cc:1081] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 1 LEADER]: Signalling peer 8f9c53697dc142e4af11d5f5e0e167de to start an election
I20260807 01:45:52.255174 25735 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "39269a566fe848f481ab0b1a50d48220"
dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
from {username='slave'} at 127.0.0.1:41584
I20260807 01:45:52.256201 25735 raft_consensus.cc:493] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:45:52.256834 25735 raft_consensus.cc:3060] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:52.260164 26039 raft_consensus.cc:993] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00: : Instructing follower 21df8eb83f204bd595e6ecd8f6405ef8 to start an election
I20260807 01:45:52.260929 26074 raft_consensus.cc:1081] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 1 LEADER]: Signalling peer 21df8eb83f204bd595e6ecd8f6405ef8 to start an election
I20260807 01:45:52.263676 25624 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a7a55cd1670e40ec94e4269dc15517bb"
dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
from {username='slave'} at 127.0.0.1:51114
I20260807 01:45:52.264273 25624 raft_consensus.cc:493] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:45:52.264611 25624 raft_consensus.cc:3060] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:52.267431 25735 raft_consensus.cc:515] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:52.270063 25735 leader_election.cc:290] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 2 election: Requested vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:52.271386 25622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39269a566fe848f481ab0b1a50d48220" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
I20260807 01:45:52.271920 25848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39269a566fe848f481ab0b1a50d48220" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b147ae01363d411980d3a99e72915a00"
I20260807 01:45:52.272414 25622 raft_consensus.cc:3060] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:52.272622 25848 raft_consensus.cc:3055] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:45:52.273125 25848 raft_consensus.cc:740] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b147ae01363d411980d3a99e72915a00, State: Running, Role: LEADER
I20260807 01:45:52.274492 25848 consensus_queue.cc:260] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:52.275830 25848 raft_consensus.cc:3060] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:52.275461 25624 raft_consensus.cc:515] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:52.279779 25624 leader_election.cc:290] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 2 election: Requested vote from peers 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:52.282119 25851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7a55cd1670e40ec94e4269dc15517bb" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b147ae01363d411980d3a99e72915a00"
I20260807 01:45:52.282913 25851 raft_consensus.cc:3055] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:45:52.284065 25851 raft_consensus.cc:740] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b147ae01363d411980d3a99e72915a00, State: Running, Role: LEADER
I20260807 01:45:52.285375 25851 consensus_queue.cc:260] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:52.287338 25851 raft_consensus.cc:3060] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:52.288133 25848 raft_consensus.cc:2468] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f9c53697dc142e4af11d5f5e0e167de in term 2.
I20260807 01:45:52.291656 25684 leader_election.cc:304] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f9c53697dc142e4af11d5f5e0e167de, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:45:52.293349 26052 raft_consensus.cc:2804] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:45:52.293871 25735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7a55cd1670e40ec94e4269dc15517bb" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:45:52.294615 25735 raft_consensus.cc:3060] T a7a55cd1670e40ec94e4269dc15517bb P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:52.294049 26052 raft_consensus.cc:697] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 LEADER]: Becoming Leader. State: Replica: 8f9c53697dc142e4af11d5f5e0e167de, State: Running, Role: LEADER
I20260807 01:45:52.295965 26052 consensus_queue.cc:237] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:52.304595 25622 raft_consensus.cc:2468] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f9c53697dc142e4af11d5f5e0e167de in term 2.
I20260807 01:45:52.308557 26074 raft_consensus.cc:993] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00: : Instructing follower 8f9c53697dc142e4af11d5f5e0e167de to start an election
I20260807 01:45:52.309208 26074 raft_consensus.cc:1081] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 1 LEADER]: Signalling peer 8f9c53697dc142e4af11d5f5e0e167de to start an election
I20260807 01:45:52.312521 25737 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "75e20a485954424c8b721f0ce6216da2"
dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
from {username='slave'} at 127.0.0.1:41584
I20260807 01:45:52.313298 25737 raft_consensus.cc:493] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:45:52.313845 25737 raft_consensus.cc:3060] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:52.318629 25471 catalog_manager.cc:5697] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de reported cstate change: term changed from 1 to 2, leader changed from b147ae01363d411980d3a99e72915a00 (127.24.4.3) to 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2). New cstate: current_term: 2 leader_uuid: "8f9c53697dc142e4af11d5f5e0e167de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:52.325482 25737 raft_consensus.cc:515] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:52.329455 25737 leader_election.cc:290] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 2 election: Requested vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:45:52.329725 25848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e20a485954424c8b721f0ce6216da2" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b147ae01363d411980d3a99e72915a00"
I20260807 01:45:52.330389 25848 raft_consensus.cc:3055] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:45:52.330771 25848 raft_consensus.cc:740] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b147ae01363d411980d3a99e72915a00, State: Running, Role: LEADER
I20260807 01:45:52.331071 25622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e20a485954424c8b721f0ce6216da2" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
I20260807 01:45:52.331779 25622 raft_consensus.cc:3060] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:52.331597 25848 consensus_queue.cc:260] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:52.327507 25851 raft_consensus.cc:2468] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 2.
I20260807 01:45:52.334934 25575 leader_election.cc:304] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:45:52.335819 26087 raft_consensus.cc:2804] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:45:52.335958 25848 raft_consensus.cc:3060] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:52.336349 26087 raft_consensus.cc:697] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 LEADER]: Becoming Leader. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Running, Role: LEADER
I20260807 01:45:52.337174 26087 consensus_queue.cc:237] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:52.338495 25735 raft_consensus.cc:2468] T a7a55cd1670e40ec94e4269dc15517bb P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 2.
I20260807 01:45:52.339493 25622 raft_consensus.cc:2468] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f9c53697dc142e4af11d5f5e0e167de in term 2.
I20260807 01:45:52.340682 25682 leader_election.cc:304] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, 8f9c53697dc142e4af11d5f5e0e167de; no voters:
I20260807 01:45:52.341533 26052 raft_consensus.cc:2804] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:45:52.342020 26052 raft_consensus.cc:697] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 LEADER]: Becoming Leader. State: Replica: 8f9c53697dc142e4af11d5f5e0e167de, State: Running, Role: LEADER
I20260807 01:45:52.342882 26052 consensus_queue.cc:237] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:52.343681 25848 raft_consensus.cc:2468] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f9c53697dc142e4af11d5f5e0e167de in term 2.
I20260807 01:45:52.351806 25471 catalog_manager.cc:5697] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 reported cstate change: term changed from 1 to 2, leader changed from b147ae01363d411980d3a99e72915a00 (127.24.4.3) to 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:52.352471 25472 catalog_manager.cc:5697] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de reported cstate change: term changed from 1 to 2, leader changed from b147ae01363d411980d3a99e72915a00 (127.24.4.3) to 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2). New cstate: current_term: 2 leader_uuid: "8f9c53697dc142e4af11d5f5e0e167de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: UNKNOWN } } }
I20260807 01:45:52.400079 25974 raft_consensus.cc:993] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8: : Instructing follower b147ae01363d411980d3a99e72915a00 to start an election
I20260807 01:45:52.400519 26087 raft_consensus.cc:1081] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 LEADER]: Signalling peer b147ae01363d411980d3a99e72915a00 to start an election
I20260807 01:45:52.402777 25848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "31f1ba5ca06f461a8b3001ca5c4c48a3"
dest_uuid: "b147ae01363d411980d3a99e72915a00"
from {username='slave'} at 127.0.0.1:36820
I20260807 01:45:52.403400 25848 raft_consensus.cc:493] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:45:52.403676 25848 raft_consensus.cc:3060] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:52.409986 25848 raft_consensus.cc:515] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:52.411854 25848 leader_election.cc:290] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 2 election: Requested vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387)
I20260807 01:45:52.412801 25622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31f1ba5ca06f461a8b3001ca5c4c48a3" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
I20260807 01:45:52.413019 25735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31f1ba5ca06f461a8b3001ca5c4c48a3" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:45:52.413496 25622 raft_consensus.cc:3055] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:45:52.413533 25735 raft_consensus.cc:3060] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:52.413985 25622 raft_consensus.cc:740] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Running, Role: LEADER
I20260807 01:45:52.414899 25622 consensus_queue.cc:260] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:52.416182 25622 raft_consensus.cc:3060] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:52.421700 25735 raft_consensus.cc:2468] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 2.
I20260807 01:45:52.423610 25799 leader_election.cc:304] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f9c53697dc142e4af11d5f5e0e167de, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:45:52.424132 25622 raft_consensus.cc:2468] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 2.
I20260807 01:45:52.424696 26039 raft_consensus.cc:2804] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:45:52.425259 26039 raft_consensus.cc:697] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00 [term 2 LEADER]: Becoming Leader. State: Replica: b147ae01363d411980d3a99e72915a00, State: Running, Role: LEADER
I20260807 01:45:52.426185 26039 consensus_queue.cc:237] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:52.434125 25471 catalog_manager.cc:5697] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00 reported cstate change: term changed from 1 to 2, leader changed from 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1) to b147ae01363d411980d3a99e72915a00 (127.24.4.3). New cstate: current_term: 2 leader_uuid: "b147ae01363d411980d3a99e72915a00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: HEALTHY } } }
I20260807 01:45:52.481504 25735 raft_consensus.cc:1275] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Refusing update from remote peer 21df8eb83f204bd595e6ecd8f6405ef8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:52.483038 26087 consensus_queue.cc:1048] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:45:52.497027 25848 raft_consensus.cc:1275] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Refusing update from remote peer 21df8eb83f204bd595e6ecd8f6405ef8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:52.498642 26112 consensus_queue.cc:1048] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:45:52.510638 25622 raft_consensus.cc:1275] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Refusing update from remote peer 8f9c53697dc142e4af11d5f5e0e167de: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:45:52.512149 26052 consensus_queue.cc:1048] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [LEADER]: Connected to new peer: Peer: permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260807 01:45:52.527021 25848 raft_consensus.cc:1275] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Refusing update from remote peer 8f9c53697dc142e4af11d5f5e0e167de: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:45:52.528617 26120 consensus_queue.cc:1048] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [LEADER]: Connected to new peer: Peer: permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:45:52.588673 25848 raft_consensus.cc:1275] T fde466361a8d40a48394033c480ecf50 P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Refusing update from remote peer 21df8eb83f204bd595e6ecd8f6405ef8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:52.590245 26087 consensus_queue.cc:1048] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:45:52.599752 25735 raft_consensus.cc:1275] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Refusing update from remote peer 21df8eb83f204bd595e6ecd8f6405ef8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:52.603274 26112 consensus_queue.cc:1048] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:45:52.717314 25848 raft_consensus.cc:1275] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Refusing update from remote peer 8f9c53697dc142e4af11d5f5e0e167de: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:52.718897 26120 consensus_queue.cc:1048] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [LEADER]: Connected to new peer: Peer: permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:45:52.728682 25622 raft_consensus.cc:1275] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Refusing update from remote peer 8f9c53697dc142e4af11d5f5e0e167de: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:52.730333 26089 consensus_queue.cc:1048] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [LEADER]: Connected to new peer: Peer: permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:45:52.751322 25848 raft_consensus.cc:1275] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Refusing update from remote peer 21df8eb83f204bd595e6ecd8f6405ef8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:52.752837 26112 consensus_queue.cc:1048] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:45:52.762107 25735 raft_consensus.cc:1275] T a7a55cd1670e40ec94e4269dc15517bb P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Refusing update from remote peer 21df8eb83f204bd595e6ecd8f6405ef8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:52.765429 26087 consensus_queue.cc:1048] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:45:52.851985 25622 raft_consensus.cc:1275] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Refusing update from remote peer 8f9c53697dc142e4af11d5f5e0e167de: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:52.853750 26126 consensus_queue.cc:1048] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [LEADER]: Connected to new peer: Peer: permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:45:52.863528 25848 raft_consensus.cc:1275] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Refusing update from remote peer 8f9c53697dc142e4af11d5f5e0e167de: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:52.866039 26126 consensus_queue.cc:1048] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [LEADER]: Connected to new peer: Peer: permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:45:52.914794 25622 raft_consensus.cc:1275] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Refusing update from remote peer b147ae01363d411980d3a99e72915a00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:52.916277 26039 consensus_queue.cc:1048] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:45:52.927232 25735 raft_consensus.cc:1275] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Refusing update from remote peer b147ae01363d411980d3a99e72915a00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:52.928668 26134 consensus_queue.cc:1048] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:45:54.920938 24592 auto_leader_rebalancer-test.cc:473] The leader distribution is
I20260807 01:45:54.921934 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:54.947410 25622 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06fd48ba8b4b49e2a0f51f1f9b8a67df"
dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
mode: GRACEFUL
new_leader_uuid: "b147ae01363d411980d3a99e72915a00"
from {username='slave'} at 127.0.0.1:35164
I20260807 01:45:54.948099 25622 raft_consensus.cc:606] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 LEADER]: Received request to transfer leadership to TS b147ae01363d411980d3a99e72915a00
I20260807 01:45:54.950907 25622 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a7b6bccb54c42b4895fdfdc6230eea9"
dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
mode: GRACEFUL
new_leader_uuid: "b147ae01363d411980d3a99e72915a00"
from {username='slave'} at 127.0.0.1:35164
I20260807 01:45:54.951360 25622 raft_consensus.cc:606] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 LEADER]: Received request to transfer leadership to TS b147ae01363d411980d3a99e72915a00
I20260807 01:45:54.952422 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260807 01:45:54.952899 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:55.171824 26141 raft_consensus.cc:993] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8: : Instructing follower b147ae01363d411980d3a99e72915a00 to start an election
I20260807 01:45:55.172609 26141 raft_consensus.cc:1081] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 LEADER]: Signalling peer b147ae01363d411980d3a99e72915a00 to start an election
I20260807 01:45:55.174527 25848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a7b6bccb54c42b4895fdfdc6230eea9"
dest_uuid: "b147ae01363d411980d3a99e72915a00"
from {username='slave'} at 127.0.0.1:36820
I20260807 01:45:55.175128 25848 raft_consensus.cc:493] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:45:55.175498 25848 raft_consensus.cc:3060] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:55.183259 25848 raft_consensus.cc:515] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:55.185441 25848 leader_election.cc:290] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 2 election: Requested vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387)
I20260807 01:45:55.186534 25622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a7b6bccb54c42b4895fdfdc6230eea9" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
I20260807 01:45:55.187393 25622 raft_consensus.cc:3055] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:45:55.187408 25735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a7b6bccb54c42b4895fdfdc6230eea9" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:45:55.187685 25622 raft_consensus.cc:740] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Running, Role: LEADER
I20260807 01:45:55.188222 25735 raft_consensus.cc:3060] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:55.188747 25622 consensus_queue.cc:260] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:55.189905 25622 raft_consensus.cc:3060] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:45:55.195889 25735 raft_consensus.cc:2468] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 2.
I20260807 01:45:55.195889 25622 raft_consensus.cc:2468] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 2.
I20260807 01:45:55.197391 25798 leader_election.cc:304] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:45:55.198397 26134 raft_consensus.cc:2804] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:45:55.199409 26134 raft_consensus.cc:697] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [term 2 LEADER]: Becoming Leader. State: Replica: b147ae01363d411980d3a99e72915a00, State: Running, Role: LEADER
I20260807 01:45:55.200335 26134 consensus_queue.cc:237] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:55.209817 25471 catalog_manager.cc:5697] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 reported cstate change: term changed from 1 to 2, leader changed from 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1) to b147ae01363d411980d3a99e72915a00 (127.24.4.3). New cstate: current_term: 2 leader_uuid: "b147ae01363d411980d3a99e72915a00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: HEALTHY } } }
I20260807 01:45:55.307852 26142 raft_consensus.cc:993] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8: : Instructing follower b147ae01363d411980d3a99e72915a00 to start an election
I20260807 01:45:55.308408 26141 raft_consensus.cc:1081] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 LEADER]: Signalling peer b147ae01363d411980d3a99e72915a00 to start an election
I20260807 01:45:55.310467 25848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06fd48ba8b4b49e2a0f51f1f9b8a67df"
dest_uuid: "b147ae01363d411980d3a99e72915a00"
from {username='slave'} at 127.0.0.1:36820
I20260807 01:45:55.311074 25848 raft_consensus.cc:493] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:45:55.311425 25848 raft_consensus.cc:3060] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:45:55.316903 25848 raft_consensus.cc:515] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:55.318542 25848 leader_election.cc:290] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 3 election: Requested vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387)
I20260807 01:45:55.319764 25735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06fd48ba8b4b49e2a0f51f1f9b8a67df" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:45:55.319907 25622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06fd48ba8b4b49e2a0f51f1f9b8a67df" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
I20260807 01:45:55.320283 25735 raft_consensus.cc:3060] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:45:55.320452 25622 raft_consensus.cc:3055] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:45:55.320762 25622 raft_consensus.cc:740] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Running, Role: LEADER
I20260807 01:45:55.321795 25622 consensus_queue.cc:260] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:55.323021 25622 raft_consensus.cc:3060] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:45:55.326391 25735 raft_consensus.cc:2468] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 3.
I20260807 01:45:55.329646 25799 leader_election.cc:304] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f9c53697dc142e4af11d5f5e0e167de, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:45:55.330576 25622 raft_consensus.cc:2468] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 3.
I20260807 01:45:55.330731 26134 raft_consensus.cc:2804] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:45:55.331568 26134 raft_consensus.cc:697] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 3 LEADER]: Becoming Leader. State: Replica: b147ae01363d411980d3a99e72915a00, State: Running, Role: LEADER
I20260807 01:45:55.332392 26134 consensus_queue.cc:237] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:45:55.341741 25471 catalog_manager.cc:5697] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 reported cstate change: term changed from 2 to 3, leader changed from 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1) to b147ae01363d411980d3a99e72915a00 (127.24.4.3). New cstate: current_term: 3 leader_uuid: "b147ae01363d411980d3a99e72915a00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: HEALTHY } } }
I20260807 01:45:55.762897 25622 raft_consensus.cc:1275] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Refusing update from remote peer b147ae01363d411980d3a99e72915a00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:55.764351 26134 consensus_queue.cc:1048] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:45:55.777172 25737 raft_consensus.cc:1275] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Refusing update from remote peer b147ae01363d411980d3a99e72915a00: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:45:55.780048 26145 consensus_queue.cc:1048] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:45:55.809466 25737 raft_consensus.cc:1275] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Refusing update from remote peer b147ae01363d411980d3a99e72915a00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:45:55.811007 26145 consensus_queue.cc:1048] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:45:55.821367 25622 raft_consensus.cc:1275] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Refusing update from remote peer b147ae01363d411980d3a99e72915a00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:45:55.824084 26145 consensus_queue.cc:1048] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:45:55.954223 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:55.960188 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:45:55.960711 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:56.961772 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:56.967900 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:45:56.968740 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:57.969820 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:57.975126 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:45:57.975595 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:58.976528 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:58.981842 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:45:58.982410 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:45:59.983740 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:45:59.990051 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:45:59.990940 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:46:00.992583 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:46:00.998327 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:46:00.998787 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:46:01.999811 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:46:02.006858 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:46:02.007712 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:46:03.008808 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:46:03.015516 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:46:03.016343 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:46:04.017889 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:46:04.024758 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:46:04.025287 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:46:05.074201 24592 auto_leader_rebalancer-test.cc:492] The leader distribution is
I20260807 01:46:05.086639 25848 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06fd48ba8b4b49e2a0f51f1f9b8a67df"
dest_uuid: "b147ae01363d411980d3a99e72915a00"
mode: GRACEFUL
new_leader_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
from {username='slave'} at 127.0.0.1:36834
I20260807 01:46:05.087874 25848 raft_consensus.cc:606] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 3 LEADER]: Received request to transfer leadership to TS 21df8eb83f204bd595e6ecd8f6405ef8
I20260807 01:46:05.101502 25848 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a7b6bccb54c42b4895fdfdc6230eea9"
dest_uuid: "b147ae01363d411980d3a99e72915a00"
mode: GRACEFUL
new_leader_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
from {username='slave'} at 127.0.0.1:36834
I20260807 01:46:05.102315 25848 raft_consensus.cc:606] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [term 2 LEADER]: Received request to transfer leadership to TS 21df8eb83f204bd595e6ecd8f6405ef8
I20260807 01:46:05.114897 25737 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39269a566fe848f481ab0b1a50d48220"
dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
mode: GRACEFUL
new_leader_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
from {username='slave'} at 127.0.0.1:41602
I20260807 01:46:05.115461 25737 raft_consensus.cc:606] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 LEADER]: Received request to transfer leadership to TS 21df8eb83f204bd595e6ecd8f6405ef8
I20260807 01:46:05.122720 25737 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75e20a485954424c8b721f0ce6216da2"
dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
mode: GRACEFUL
new_leader_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
from {username='slave'} at 127.0.0.1:41602
I20260807 01:46:05.123203 25737 raft_consensus.cc:606] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 LEADER]: Received request to transfer leadership to TS 21df8eb83f204bd595e6ecd8f6405ef8
I20260807 01:46:05.131068 25737 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "061b319309704115ac6bc2716e08fd4a"
dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
mode: GRACEFUL
new_leader_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
from {username='slave'} at 127.0.0.1:41602
I20260807 01:46:05.131641 25737 raft_consensus.cc:606] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 1 LEADER]: Received request to transfer leadership to TS 21df8eb83f204bd595e6ecd8f6405ef8
I20260807 01:46:05.187549 26175 raft_consensus.cc:993] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de: : Instructing follower 21df8eb83f204bd595e6ecd8f6405ef8 to start an election
I20260807 01:46:05.189044 26180 raft_consensus.cc:1081] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 LEADER]: Signalling peer 21df8eb83f204bd595e6ecd8f6405ef8 to start an election
I20260807 01:46:05.192562 25622 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "39269a566fe848f481ab0b1a50d48220"
dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
from {username='slave'} at 127.0.0.1:51124
I20260807 01:46:05.193225 25622 raft_consensus.cc:493] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:46:05.193706 25622 raft_consensus.cc:3060] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:05.208284 25622 raft_consensus.cc:515] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:05.212121 25622 leader_election.cc:290] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 3 election: Requested vote from peers 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:46:05.213332 25737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39269a566fe848f481ab0b1a50d48220" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:46:05.214594 25851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39269a566fe848f481ab0b1a50d48220" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b147ae01363d411980d3a99e72915a00"
I20260807 01:46:05.214597 25737 raft_consensus.cc:3055] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:46:05.215109 25851 raft_consensus.cc:3060] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:05.215093 25737 raft_consensus.cc:740] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8f9c53697dc142e4af11d5f5e0e167de, State: Running, Role: LEADER
I20260807 01:46:05.216342 25737 consensus_queue.cc:260] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:05.217898 25737 raft_consensus.cc:3060] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:05.224360 25851 raft_consensus.cc:2468] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 3.
I20260807 01:46:05.227607 25737 raft_consensus.cc:2468] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 3.
I20260807 01:46:05.227066 25575 leader_election.cc:304] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:46:05.230417 26179 raft_consensus.cc:2804] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:05.231899 26179 raft_consensus.cc:697] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 LEADER]: Becoming Leader. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Running, Role: LEADER
I20260807 01:46:05.233595 26179 consensus_queue.cc:237] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:05.248227 25471 catalog_manager.cc:5697] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 reported cstate change: term changed from 2 to 3, leader changed from 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2) to 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1). New cstate: current_term: 3 leader_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:05.256008 26180 raft_consensus.cc:993] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de: : Instructing follower 21df8eb83f204bd595e6ecd8f6405ef8 to start an election
I20260807 01:46:05.256819 26175 raft_consensus.cc:1081] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 LEADER]: Signalling peer 21df8eb83f204bd595e6ecd8f6405ef8 to start an election
I20260807 01:46:05.259465 25622 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "75e20a485954424c8b721f0ce6216da2"
dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
from {username='slave'} at 127.0.0.1:51124
I20260807 01:46:05.259929 25622 raft_consensus.cc:493] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:46:05.260200 25622 raft_consensus.cc:3060] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:05.268913 25622 raft_consensus.cc:515] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:05.271308 25622 leader_election.cc:290] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 3 election: Requested vote from peers 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:46:05.272220 25737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e20a485954424c8b721f0ce6216da2" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:46:05.272548 25848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e20a485954424c8b721f0ce6216da2" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b147ae01363d411980d3a99e72915a00"
I20260807 01:46:05.272854 25737 raft_consensus.cc:3055] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:46:05.273070 25848 raft_consensus.cc:3060] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:05.273257 25737 raft_consensus.cc:740] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8f9c53697dc142e4af11d5f5e0e167de, State: Running, Role: LEADER
I20260807 01:46:05.274588 25737 consensus_queue.cc:260] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:05.276111 25737 raft_consensus.cc:3060] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:05.282122 25737 raft_consensus.cc:2468] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 3.
I20260807 01:46:05.284482 25573 leader_election.cc:304] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, 8f9c53697dc142e4af11d5f5e0e167de; no voters:
I20260807 01:46:05.285579 26179 raft_consensus.cc:2804] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:05.285661 25848 raft_consensus.cc:2468] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 3.
I20260807 01:46:05.286453 26179 raft_consensus.cc:697] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 LEADER]: Becoming Leader. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Running, Role: LEADER
I20260807 01:46:05.288707 26179 consensus_queue.cc:237] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:05.301146 25471 catalog_manager.cc:5697] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 reported cstate change: term changed from 2 to 3, leader changed from 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2) to 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1). New cstate: current_term: 3 leader_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:05.394766 26180 raft_consensus.cc:993] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de: : Instructing follower 21df8eb83f204bd595e6ecd8f6405ef8 to start an election
I20260807 01:46:05.395573 26180 raft_consensus.cc:1081] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 1 LEADER]: Signalling peer 21df8eb83f204bd595e6ecd8f6405ef8 to start an election
I20260807 01:46:05.398084 25622 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "061b319309704115ac6bc2716e08fd4a"
dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
from {username='slave'} at 127.0.0.1:51124
I20260807 01:46:05.398655 25622 raft_consensus.cc:493] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:46:05.398991 25622 raft_consensus.cc:3060] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:05.405894 25622 raft_consensus.cc:515] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:05.408082 25622 leader_election.cc:290] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 2 election: Requested vote from peers 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:46:05.408986 25737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061b319309704115ac6bc2716e08fd4a" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:46:05.409538 25848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061b319309704115ac6bc2716e08fd4a" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b147ae01363d411980d3a99e72915a00"
I20260807 01:46:05.409734 25737 raft_consensus.cc:3055] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:05.410135 25848 raft_consensus.cc:3060] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:05.410212 25737 raft_consensus.cc:740] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f9c53697dc142e4af11d5f5e0e167de, State: Running, Role: LEADER
I20260807 01:46:05.411504 25737 consensus_queue.cc:260] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:05.413125 25737 raft_consensus.cc:3060] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:05.414098 26168 raft_consensus.cc:993] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00: : Instructing follower 21df8eb83f204bd595e6ecd8f6405ef8 to start an election
I20260807 01:46:05.415639 26189 raft_consensus.cc:1081] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 3 LEADER]: Signalling peer 21df8eb83f204bd595e6ecd8f6405ef8 to start an election
I20260807 01:46:05.417737 25848 raft_consensus.cc:2468] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 2.
I20260807 01:46:05.418587 25622 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06fd48ba8b4b49e2a0f51f1f9b8a67df"
dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
from {username='slave'} at 127.0.0.1:51114
I20260807 01:46:05.419154 25622 raft_consensus.cc:493] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:46:05.419507 25622 raft_consensus.cc:3060] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:05.419472 25575 leader_election.cc:304] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:46:05.421109 26179 raft_consensus.cc:2804] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:46:05.422678 25737 raft_consensus.cc:2468] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 2.
I20260807 01:46:05.421976 26179 raft_consensus.cc:697] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 LEADER]: Becoming Leader. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Running, Role: LEADER
I20260807 01:46:05.424357 26179 consensus_queue.cc:237] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:05.430353 25622 raft_consensus.cc:515] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:05.432551 25622 leader_election.cc:290] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 4 election: Requested vote from peers 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:46:05.434685 25737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06fd48ba8b4b49e2a0f51f1f9b8a67df" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:46:05.435350 25737 raft_consensus.cc:3060] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:05.440742 25471 catalog_manager.cc:5697] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 reported cstate change: term changed from 1 to 2, leader changed from 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2) to 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:05.442826 26168 raft_consensus.cc:993] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00: : Instructing follower 21df8eb83f204bd595e6ecd8f6405ef8 to start an election
I20260807 01:46:05.443507 26189 raft_consensus.cc:1081] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [term 2 LEADER]: Signalling peer 21df8eb83f204bd595e6ecd8f6405ef8 to start an election
I20260807 01:46:05.443825 25848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06fd48ba8b4b49e2a0f51f1f9b8a67df" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b147ae01363d411980d3a99e72915a00"
I20260807 01:46:05.444147 25737 raft_consensus.cc:2468] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 4.
I20260807 01:46:05.444517 25848 raft_consensus.cc:3055] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:46:05.444911 25848 raft_consensus.cc:740] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b147ae01363d411980d3a99e72915a00, State: Running, Role: LEADER
I20260807 01:46:05.445413 25573 leader_election.cc:304] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, 8f9c53697dc142e4af11d5f5e0e167de; no voters:
I20260807 01:46:05.446098 25848 consensus_queue.cc:260] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:05.446792 25622 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a7b6bccb54c42b4895fdfdc6230eea9"
dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
from {username='slave'} at 127.0.0.1:51114
I20260807 01:46:05.447333 25622 raft_consensus.cc:493] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:46:05.447705 25622 raft_consensus.cc:3060] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:05.447862 25848 raft_consensus.cc:3060] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:05.446465 26179 raft_consensus.cc:2804] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:46:05.450229 26179 raft_consensus.cc:697] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 4 LEADER]: Becoming Leader. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Running, Role: LEADER
I20260807 01:46:05.451939 26179 consensus_queue.cc:237] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:05.458567 25848 raft_consensus.cc:2468] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 4.
I20260807 01:46:05.458297 25622 raft_consensus.cc:515] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:05.460566 25622 leader_election.cc:290] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 3 election: Requested vote from peers 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:46:05.462371 25848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a7b6bccb54c42b4895fdfdc6230eea9" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b147ae01363d411980d3a99e72915a00"
I20260807 01:46:05.463048 25848 raft_consensus.cc:3055] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:46:05.463297 25737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a7b6bccb54c42b4895fdfdc6230eea9" candidate_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:46:05.463443 25848 raft_consensus.cc:740] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b147ae01363d411980d3a99e72915a00, State: Running, Role: LEADER
I20260807 01:46:05.463908 25737 raft_consensus.cc:3060] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:05.464823 25848 consensus_queue.cc:260] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:05.466354 25848 raft_consensus.cc:3060] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:05.469527 25471 catalog_manager.cc:5697] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 reported cstate change: term changed from 3 to 4, leader changed from b147ae01363d411980d3a99e72915a00 (127.24.4.3) to 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1). New cstate: current_term: 4 leader_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:05.471055 25737 raft_consensus.cc:2468] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 3.
I20260807 01:46:05.472440 25573 leader_election.cc:304] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21df8eb83f204bd595e6ecd8f6405ef8, 8f9c53697dc142e4af11d5f5e0e167de; no voters:
I20260807 01:46:05.473239 26179 raft_consensus.cc:2804] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:05.473812 25848 raft_consensus.cc:2468] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21df8eb83f204bd595e6ecd8f6405ef8 in term 3.
I20260807 01:46:05.473875 26179 raft_consensus.cc:697] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 LEADER]: Becoming Leader. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Running, Role: LEADER
I20260807 01:46:05.476470 26179 consensus_queue.cc:237] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:05.489130 25471 catalog_manager.cc:5697] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 reported cstate change: term changed from 2 to 3, leader changed from b147ae01363d411980d3a99e72915a00 (127.24.4.3) to 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1). New cstate: current_term: 3 leader_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:05.682473 25848 raft_consensus.cc:1275] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Refusing update from remote peer 21df8eb83f204bd595e6ecd8f6405ef8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:46:05.684202 26179 consensus_queue.cc:1048] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:46:05.707731 25737 raft_consensus.cc:1275] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Refusing update from remote peer 21df8eb83f204bd595e6ecd8f6405ef8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:46:05.710472 26172 consensus_queue.cc:1048] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260807 01:46:05.784885 25737 raft_consensus.cc:1275] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Refusing update from remote peer 21df8eb83f204bd595e6ecd8f6405ef8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:46:05.787431 26201 consensus_queue.cc:1048] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:46:05.803922 25848 raft_consensus.cc:1275] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Refusing update from remote peer 21df8eb83f204bd595e6ecd8f6405ef8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:46:05.805965 26201 consensus_queue.cc:1048] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:46:05.958039 25848 raft_consensus.cc:1275] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Refusing update from remote peer 21df8eb83f204bd595e6ecd8f6405ef8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:46:05.958418 25851 raft_consensus.cc:1275] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 4 FOLLOWER]: Refusing update from remote peer 21df8eb83f204bd595e6ecd8f6405ef8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:46:05.959671 26201 consensus_queue.cc:1048] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:46:05.959652 26179 consensus_queue.cc:1048] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:46:05.968925 25851 raft_consensus.cc:1275] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Refusing update from remote peer 21df8eb83f204bd595e6ecd8f6405ef8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:46:05.972985 25737 raft_consensus.cc:1275] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Refusing update from remote peer 21df8eb83f204bd595e6ecd8f6405ef8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:46:05.974794 26199 consensus_queue.cc:1048] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:05.975950 25735 raft_consensus.cc:1275] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Refusing update from remote peer 21df8eb83f204bd595e6ecd8f6405ef8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:46:05.978271 26179 consensus_queue.cc:1048] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:05.983716 26172 consensus_queue.cc:1048] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:46:05.987002 25737 raft_consensus.cc:1275] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de [term 4 FOLLOWER]: Refusing update from remote peer 21df8eb83f204bd595e6ecd8f6405ef8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:46:05.989539 26172 consensus_queue.cc:1048] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:46:08.195125 24592 auto_leader_rebalancer-test.cc:507] The leader distribution is
I20260807 01:46:08.195912 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:46:08.203536 25622 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "061b319309704115ac6bc2716e08fd4a"
dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
mode: GRACEFUL
new_leader_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
from {username='slave'} at 127.0.0.1:35164
I20260807 01:46:08.204067 25622 raft_consensus.cc:606] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 LEADER]: Received request to transfer leadership to TS 8f9c53697dc142e4af11d5f5e0e167de
I20260807 01:46:08.207712 25622 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39269a566fe848f481ab0b1a50d48220"
dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
mode: GRACEFUL
new_leader_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
from {username='slave'} at 127.0.0.1:35164
I20260807 01:46:08.208555 25622 raft_consensus.cc:606] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 LEADER]: Received request to transfer leadership to TS 8f9c53697dc142e4af11d5f5e0e167de
I20260807 01:46:08.212069 25622 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75e20a485954424c8b721f0ce6216da2"
dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
mode: GRACEFUL
new_leader_uuid: "b147ae01363d411980d3a99e72915a00"
from {username='slave'} at 127.0.0.1:35164
I20260807 01:46:08.212595 25622 raft_consensus.cc:606] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 LEADER]: Received request to transfer leadership to TS b147ae01363d411980d3a99e72915a00
I20260807 01:46:08.215065 25622 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a7a55cd1670e40ec94e4269dc15517bb"
dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
mode: GRACEFUL
new_leader_uuid: "b147ae01363d411980d3a99e72915a00"
from {username='slave'} at 127.0.0.1:35164
I20260807 01:46:08.215540 25622 raft_consensus.cc:606] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 LEADER]: Received request to transfer leadership to TS b147ae01363d411980d3a99e72915a00
I20260807 01:46:08.216778 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260807 01:46:08.217427 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:46:08.218557 26172 raft_consensus.cc:993] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8: : Instructing follower 8f9c53697dc142e4af11d5f5e0e167de to start an election
I20260807 01:46:08.219074 26224 raft_consensus.cc:1081] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 LEADER]: Signalling peer 8f9c53697dc142e4af11d5f5e0e167de to start an election
I20260807 01:46:08.220710 25734 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "39269a566fe848f481ab0b1a50d48220"
dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
from {username='slave'} at 127.0.0.1:41600
I20260807 01:46:08.221190 25734 raft_consensus.cc:493] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:46:08.221643 25734 raft_consensus.cc:3060] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:08.227797 25734 raft_consensus.cc:515] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:08.230003 25734 leader_election.cc:290] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 4 election: Requested vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:46:08.231388 25848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39269a566fe848f481ab0b1a50d48220" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b147ae01363d411980d3a99e72915a00"
I20260807 01:46:08.231494 25622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39269a566fe848f481ab0b1a50d48220" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
I20260807 01:46:08.231891 25848 raft_consensus.cc:3060] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:08.232290 25622 raft_consensus.cc:3055] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:46:08.232580 25622 raft_consensus.cc:740] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Running, Role: LEADER
I20260807 01:46:08.233197 25622 consensus_queue.cc:260] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:08.234112 25622 raft_consensus.cc:3060] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:08.238021 25848 raft_consensus.cc:2468] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f9c53697dc142e4af11d5f5e0e167de in term 4.
I20260807 01:46:08.238695 25622 raft_consensus.cc:2468] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f9c53697dc142e4af11d5f5e0e167de in term 4.
I20260807 01:46:08.239526 25684 leader_election.cc:304] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f9c53697dc142e4af11d5f5e0e167de, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:46:08.240363 26223 raft_consensus.cc:2804] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:46:08.241310 26223 raft_consensus.cc:697] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 4 LEADER]: Becoming Leader. State: Replica: 8f9c53697dc142e4af11d5f5e0e167de, State: Running, Role: LEADER
I20260807 01:46:08.242067 26223 consensus_queue.cc:237] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:08.249924 26172 raft_consensus.cc:993] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8: : Instructing follower b147ae01363d411980d3a99e72915a00 to start an election
I20260807 01:46:08.250391 26224 raft_consensus.cc:1081] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 LEADER]: Signalling peer b147ae01363d411980d3a99e72915a00 to start an election
I20260807 01:46:08.250871 25472 catalog_manager.cc:5697] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de reported cstate change: term changed from 3 to 4, leader changed from 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1) to 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2). New cstate: current_term: 4 leader_uuid: "8f9c53697dc142e4af11d5f5e0e167de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:08.251755 25848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a7a55cd1670e40ec94e4269dc15517bb"
dest_uuid: "b147ae01363d411980d3a99e72915a00"
from {username='slave'} at 127.0.0.1:36820
I20260807 01:46:08.252202 25848 raft_consensus.cc:493] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:46:08.252532 25848 raft_consensus.cc:3060] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:08.259044 25848 raft_consensus.cc:515] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:08.261914 25848 leader_election.cc:290] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 3 election: Requested vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387)
I20260807 01:46:08.262382 25622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7a55cd1670e40ec94e4269dc15517bb" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
I20260807 01:46:08.262576 25734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7a55cd1670e40ec94e4269dc15517bb" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:46:08.262974 25622 raft_consensus.cc:3055] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:46:08.263285 25622 raft_consensus.cc:740] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Running, Role: LEADER
I20260807 01:46:08.263168 25734 raft_consensus.cc:3060] T a7a55cd1670e40ec94e4269dc15517bb P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:08.264320 25622 consensus_queue.cc:260] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:08.265676 25622 raft_consensus.cc:3060] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:08.269521 25734 raft_consensus.cc:2468] T a7a55cd1670e40ec94e4269dc15517bb P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 3.
I20260807 01:46:08.270254 25622 raft_consensus.cc:2468] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 3.
I20260807 01:46:08.270912 25799 leader_election.cc:304] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f9c53697dc142e4af11d5f5e0e167de, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:46:08.271627 26189 raft_consensus.cc:2804] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:08.272796 26189 raft_consensus.cc:697] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 3 LEADER]: Becoming Leader. State: Replica: b147ae01363d411980d3a99e72915a00, State: Running, Role: LEADER
I20260807 01:46:08.273836 26189 consensus_queue.cc:237] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:08.281338 26172 raft_consensus.cc:993] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8: : Instructing follower 8f9c53697dc142e4af11d5f5e0e167de to start an election
I20260807 01:46:08.281785 26224 raft_consensus.cc:1081] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 LEADER]: Signalling peer 8f9c53697dc142e4af11d5f5e0e167de to start an election
I20260807 01:46:08.281256 25472 catalog_manager.cc:5697] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 reported cstate change: term changed from 2 to 3, leader changed from 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1) to b147ae01363d411980d3a99e72915a00 (127.24.4.3). New cstate: current_term: 3 leader_uuid: "b147ae01363d411980d3a99e72915a00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:08.283766 25734 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "061b319309704115ac6bc2716e08fd4a"
dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
from {username='slave'} at 127.0.0.1:41600
I20260807 01:46:08.284279 25734 raft_consensus.cc:493] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:46:08.284857 25734 raft_consensus.cc:3060] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:08.291265 25734 raft_consensus.cc:515] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:08.293111 25734 leader_election.cc:290] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 3 election: Requested vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), b147ae01363d411980d3a99e72915a00 (127.24.4.3:39743)
I20260807 01:46:08.294063 25622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061b319309704115ac6bc2716e08fd4a" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
I20260807 01:46:08.294278 25851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "061b319309704115ac6bc2716e08fd4a" candidate_uuid: "8f9c53697dc142e4af11d5f5e0e167de" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b147ae01363d411980d3a99e72915a00"
I20260807 01:46:08.294687 25622 raft_consensus.cc:3055] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:46:08.295011 25851 raft_consensus.cc:3060] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:08.295028 25622 raft_consensus.cc:740] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Running, Role: LEADER
I20260807 01:46:08.295912 25622 consensus_queue.cc:260] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:08.296749 25622 raft_consensus.cc:3060] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:08.300405 25851 raft_consensus.cc:2468] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f9c53697dc142e4af11d5f5e0e167de in term 3.
I20260807 01:46:08.301378 25684 leader_election.cc:304] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f9c53697dc142e4af11d5f5e0e167de, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:46:08.301520 25622 raft_consensus.cc:2468] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f9c53697dc142e4af11d5f5e0e167de in term 3.
I20260807 01:46:08.302075 26223 raft_consensus.cc:2804] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:08.302580 26223 raft_consensus.cc:697] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 3 LEADER]: Becoming Leader. State: Replica: 8f9c53697dc142e4af11d5f5e0e167de, State: Running, Role: LEADER
I20260807 01:46:08.303359 26223 consensus_queue.cc:237] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:08.310920 25472 catalog_manager.cc:5697] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de reported cstate change: term changed from 2 to 3, leader changed from 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1) to 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2). New cstate: current_term: 3 leader_uuid: "8f9c53697dc142e4af11d5f5e0e167de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:08.444311 26172 raft_consensus.cc:993] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8: : Instructing follower b147ae01363d411980d3a99e72915a00 to start an election
I20260807 01:46:08.445077 26224 raft_consensus.cc:1081] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 LEADER]: Signalling peer b147ae01363d411980d3a99e72915a00 to start an election
I20260807 01:46:08.446763 25851 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "75e20a485954424c8b721f0ce6216da2"
dest_uuid: "b147ae01363d411980d3a99e72915a00"
from {username='slave'} at 127.0.0.1:36820
I20260807 01:46:08.447400 25851 raft_consensus.cc:493] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:46:08.447696 25851 raft_consensus.cc:3060] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:08.452625 25851 raft_consensus.cc:515] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:08.454223 25851 leader_election.cc:290] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 4 election: Requested vote from peers 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955), 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387)
I20260807 01:46:08.455153 25622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e20a485954424c8b721f0ce6216da2" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "21df8eb83f204bd595e6ecd8f6405ef8"
I20260807 01:46:08.455153 25734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e20a485954424c8b721f0ce6216da2" candidate_uuid: "b147ae01363d411980d3a99e72915a00" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8f9c53697dc142e4af11d5f5e0e167de"
I20260807 01:46:08.455891 25622 raft_consensus.cc:3055] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:46:08.455952 25734 raft_consensus.cc:3060] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:08.456290 25622 raft_consensus.cc:740] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 21df8eb83f204bd595e6ecd8f6405ef8, State: Running, Role: LEADER
I20260807 01:46:08.457032 25622 consensus_queue.cc:260] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:08.458207 25622 raft_consensus.cc:3060] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:08.463274 25734 raft_consensus.cc:2468] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 4.
I20260807 01:46:08.464788 25622 raft_consensus.cc:2468] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b147ae01363d411980d3a99e72915a00 in term 4.
I20260807 01:46:08.465031 25799 leader_election.cc:304] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f9c53697dc142e4af11d5f5e0e167de, b147ae01363d411980d3a99e72915a00; no voters:
I20260807 01:46:08.466636 26189 raft_consensus.cc:2804] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:46:08.467077 26189 raft_consensus.cc:697] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 4 LEADER]: Becoming Leader. State: Replica: b147ae01363d411980d3a99e72915a00, State: Running, Role: LEADER
I20260807 01:46:08.468025 26189 consensus_queue.cc:237] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } }
I20260807 01:46:08.477541 25472 catalog_manager.cc:5697] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 reported cstate change: term changed from 3 to 4, leader changed from 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1) to b147ae01363d411980d3a99e72915a00 (127.24.4.3). New cstate: current_term: 4 leader_uuid: "b147ae01363d411980d3a99e72915a00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:08.693013 25622 raft_consensus.cc:1275] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Refusing update from remote peer 8f9c53697dc142e4af11d5f5e0e167de: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:46:08.694453 26223 consensus_queue.cc:1048] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [LEADER]: Connected to new peer: Peer: permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:46:08.705870 25851 raft_consensus.cc:1275] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Refusing update from remote peer 8f9c53697dc142e4af11d5f5e0e167de: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:46:08.708014 26223 consensus_queue.cc:1048] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [LEADER]: Connected to new peer: Peer: permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:46:08.738488 25851 raft_consensus.cc:1275] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 4 FOLLOWER]: Refusing update from remote peer 8f9c53697dc142e4af11d5f5e0e167de: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:46:08.739797 26223 consensus_queue.cc:1048] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [LEADER]: Connected to new peer: Peer: permanent_uuid: "b147ae01363d411980d3a99e72915a00" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:46:08.746989 25734 raft_consensus.cc:1275] T a7a55cd1670e40ec94e4269dc15517bb P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Refusing update from remote peer b147ae01363d411980d3a99e72915a00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:46:08.748404 26189 consensus_queue.cc:1048] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:46:08.751979 25622 raft_consensus.cc:1275] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 4 FOLLOWER]: Refusing update from remote peer 8f9c53697dc142e4af11d5f5e0e167de: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:46:08.754256 26253 consensus_queue.cc:1048] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [LEADER]: Connected to new peer: Peer: permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:46:08.766032 25622 raft_consensus.cc:1275] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Refusing update from remote peer b147ae01363d411980d3a99e72915a00: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:46:08.768141 26236 consensus_queue.cc:1048] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:46:08.923326 25734 raft_consensus.cc:1275] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [term 4 FOLLOWER]: Refusing update from remote peer b147ae01363d411980d3a99e72915a00: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:46:08.924949 26257 consensus_queue.cc:1048] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9c53697dc142e4af11d5f5e0e167de" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:46:08.936246 25622 raft_consensus.cc:1275] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 4 FOLLOWER]: Refusing update from remote peer b147ae01363d411980d3a99e72915a00: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:46:08.939141 26257 consensus_queue.cc:1048] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21df8eb83f204bd595e6ecd8f6405ef8" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:46:09.218938 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:46:09.224927 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:46:09.225600 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:46:10.227231 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:46:10.235153 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:46:10.235837 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:46:11.236843 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:46:11.242355 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:46:11.243333 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:46:12.244488 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:46:12.250272 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:46:12.250782 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:46:13.252043 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:46:13.257977 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:46:13.258555 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:46:14.259907 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:46:14.266999 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:46:14.267561 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:46:15.268852 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:46:15.275560 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:46:15.276083 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:46:16.277109 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:46:16.283043 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:46:16.283592 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:46:17.284665 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:46:17.289924 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:46:17.290458 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:46:18.338975 24592 auto_leader_rebalancer-test.cc:518] The leader distribution is
I20260807 01:46:18.339843 24592 tablet_server.cc:179] TabletServer@127.24.4.1:0 shutting down...
W20260807 01:46:18.361519 25798 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 5 similar messages]
W20260807 01:46:18.370692 25798 consensus_peers.cc:597] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 -> Peer 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955): Couldn't send request to peer 21df8eb83f204bd595e6ecd8f6405ef8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260807 01:46:18.381299 25682 consensus_peers.cc:597] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de -> Peer 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955): Couldn't send request to peer 21df8eb83f204bd595e6ecd8f6405ef8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:18.395361 24592 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260807 01:46:18.396209 24592 tablet_replica.cc:333] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8: stopping tablet replica
I20260807 01:46:18.397022 24592 raft_consensus.cc:2243] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:18.397787 24592 raft_consensus.cc:2272] T 39269a566fe848f481ab0b1a50d48220 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.400054 24592 tablet_replica.cc:333] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8: stopping tablet replica
I20260807 01:46:18.400707 24592 raft_consensus.cc:2243] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:18.401183 24592 raft_consensus.cc:2272] T 061b319309704115ac6bc2716e08fd4a P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.403237 24592 tablet_replica.cc:333] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8: stopping tablet replica
I20260807 01:46:18.403813 24592 raft_consensus.cc:2243] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:18.404250 24592 raft_consensus.cc:2272] T eff0dbafbd3f457ba6c59d0e8bf16705 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.407591 24592 tablet_replica.cc:333] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8: stopping tablet replica
I20260807 01:46:18.408567 24592 raft_consensus.cc:2243] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:18.409286 24592 raft_consensus.cc:2272] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.411774 24592 tablet_replica.cc:333] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8: stopping tablet replica
I20260807 01:46:18.412312 24592 raft_consensus.cc:2243] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 LEADER]: Raft consensus shutting down.
I20260807 01:46:18.413121 24592 raft_consensus.cc:2272] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.415019 24592 tablet_replica.cc:333] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8: stopping tablet replica
I20260807 01:46:18.415524 24592 raft_consensus.cc:2243] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:18.415920 24592 raft_consensus.cc:2272] T a7a55cd1670e40ec94e4269dc15517bb P 21df8eb83f204bd595e6ecd8f6405ef8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.417718 24592 tablet_replica.cc:333] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8: stopping tablet replica
I20260807 01:46:18.418222 24592 raft_consensus.cc:2243] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 LEADER]: Raft consensus shutting down.
I20260807 01:46:18.419013 24592 raft_consensus.cc:2272] T fde466361a8d40a48394033c480ecf50 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.420878 24592 tablet_replica.cc:333] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8: stopping tablet replica
I20260807 01:46:18.421365 24592 raft_consensus.cc:2243] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 LEADER]: Raft consensus shutting down.
I20260807 01:46:18.422143 24592 raft_consensus.cc:2272] T a8ebd958223c4616b2055f66cfcb5581 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.423990 24592 tablet_replica.cc:333] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8: stopping tablet replica
I20260807 01:46:18.424463 24592 raft_consensus.cc:2243] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:18.424923 24592 raft_consensus.cc:2272] T 75e20a485954424c8b721f0ce6216da2 P 21df8eb83f204bd595e6ecd8f6405ef8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.426833 24592 tablet_replica.cc:333] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8: stopping tablet replica
I20260807 01:46:18.427322 24592 raft_consensus.cc:2243] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 4 LEADER]: Raft consensus shutting down.
I20260807 01:46:18.428372 24592 raft_consensus.cc:2272] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 21df8eb83f204bd595e6ecd8f6405ef8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.466796 24592 tablet_server.cc:196] TabletServer@127.24.4.1:0 shutdown complete.
W20260807 01:46:18.486380 25798 consensus_peers.cc:597] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 -> Peer 21df8eb83f204bd595e6ecd8f6405ef8 (127.24.4.1:33955): Couldn't send request to peer 21df8eb83f204bd595e6ecd8f6405ef8. Status: Network error: Client connection negotiation failed: client connection to 127.24.4.1:33955: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:18.495617 24592 tablet_server.cc:179] TabletServer@127.24.4.2:0 shutting down...
I20260807 01:46:18.532814 24592 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260807 01:46:18.533527 24592 tablet_replica.cc:333] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 8f9c53697dc142e4af11d5f5e0e167de: stopping tablet replica
I20260807 01:46:18.534530 24592 raft_consensus.cc:2243] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:18.535192 24592 raft_consensus.cc:2272] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.537392 24592 tablet_replica.cc:333] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de: stopping tablet replica
I20260807 01:46:18.537977 24592 raft_consensus.cc:2243] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 3 LEADER]: Raft consensus shutting down.
I20260807 01:46:18.538839 24592 raft_consensus.cc:2272] T eff0dbafbd3f457ba6c59d0e8bf16705 P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.540648 24592 tablet_replica.cc:333] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de: stopping tablet replica
I20260807 01:46:18.541160 24592 raft_consensus.cc:2243] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 4 LEADER]: Raft consensus shutting down.
I20260807 01:46:18.541985 24592 raft_consensus.cc:2272] T 39269a566fe848f481ab0b1a50d48220 P 8f9c53697dc142e4af11d5f5e0e167de [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.543835 24592 tablet_replica.cc:333] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de: stopping tablet replica
I20260807 01:46:18.544353 24592 raft_consensus.cc:2243] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 3 LEADER]: Raft consensus shutting down.
I20260807 01:46:18.545128 24592 raft_consensus.cc:2272] T 061b319309704115ac6bc2716e08fd4a P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.547046 24592 tablet_replica.cc:333] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de: stopping tablet replica
I20260807 01:46:18.547551 24592 raft_consensus.cc:2243] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:18.547987 24592 raft_consensus.cc:2272] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P 8f9c53697dc142e4af11d5f5e0e167de [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.549678 24592 tablet_replica.cc:333] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de: stopping tablet replica
I20260807 01:46:18.550241 24592 raft_consensus.cc:2243] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:18.550709 24592 raft_consensus.cc:2272] T 75e20a485954424c8b721f0ce6216da2 P 8f9c53697dc142e4af11d5f5e0e167de [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.552552 24592 tablet_replica.cc:333] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 8f9c53697dc142e4af11d5f5e0e167de: stopping tablet replica
I20260807 01:46:18.553081 24592 raft_consensus.cc:2243] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:18.553571 24592 raft_consensus.cc:2272] T 5a7b6bccb54c42b4895fdfdc6230eea9 P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.555572 24592 tablet_replica.cc:333] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de: stopping tablet replica
I20260807 01:46:18.556124 24592 raft_consensus.cc:2243] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:18.556567 24592 raft_consensus.cc:2272] T a8ebd958223c4616b2055f66cfcb5581 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.558430 24592 tablet_replica.cc:333] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de: stopping tablet replica
I20260807 01:46:18.559106 24592 raft_consensus.cc:2243] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:18.560026 24592 raft_consensus.cc:2272] T fde466361a8d40a48394033c480ecf50 P 8f9c53697dc142e4af11d5f5e0e167de [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.562038 24592 tablet_replica.cc:333] T a7a55cd1670e40ec94e4269dc15517bb P 8f9c53697dc142e4af11d5f5e0e167de: stopping tablet replica
I20260807 01:46:18.562556 24592 raft_consensus.cc:2243] T a7a55cd1670e40ec94e4269dc15517bb P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:18.563026 24592 raft_consensus.cc:2272] T a7a55cd1670e40ec94e4269dc15517bb P 8f9c53697dc142e4af11d5f5e0e167de [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.596876 24592 tablet_server.cc:196] TabletServer@127.24.4.2:0 shutdown complete.
I20260807 01:46:18.620535 24592 tablet_server.cc:179] TabletServer@127.24.4.3:0 shutting down...
W20260807 01:46:18.654521 25799 consensus_peers.cc:597] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00 -> Peer 8f9c53697dc142e4af11d5f5e0e167de (127.24.4.2:36387): Couldn't send request to peer 8f9c53697dc142e4af11d5f5e0e167de. Status: Network error: Client connection negotiation failed: client connection to 127.24.4.2:36387: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:18.654695 24592 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260807 01:46:18.655421 24592 tablet_replica.cc:333] T fde466361a8d40a48394033c480ecf50 P b147ae01363d411980d3a99e72915a00: stopping tablet replica
I20260807 01:46:18.656091 24592 raft_consensus.cc:2243] T fde466361a8d40a48394033c480ecf50 P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:18.656652 24592 raft_consensus.cc:2272] T fde466361a8d40a48394033c480ecf50 P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.658787 24592 tablet_replica.cc:333] T a8ebd958223c4616b2055f66cfcb5581 P b147ae01363d411980d3a99e72915a00: stopping tablet replica
I20260807 01:46:18.659314 24592 raft_consensus.cc:2243] T a8ebd958223c4616b2055f66cfcb5581 P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:18.659772 24592 raft_consensus.cc:2272] T a8ebd958223c4616b2055f66cfcb5581 P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.661607 24592 tablet_replica.cc:333] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00: stopping tablet replica
I20260807 01:46:18.662143 24592 raft_consensus.cc:2243] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:18.662664 24592 raft_consensus.cc:2272] T 5a7b6bccb54c42b4895fdfdc6230eea9 P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.664489 24592 tablet_replica.cc:333] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00: stopping tablet replica
I20260807 01:46:18.665035 24592 raft_consensus.cc:2243] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00 [term 2 LEADER]: Raft consensus shutting down.
I20260807 01:46:18.665922 24592 raft_consensus.cc:2272] T 31f1ba5ca06f461a8b3001ca5c4c48a3 P b147ae01363d411980d3a99e72915a00 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.667929 24592 tablet_replica.cc:333] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00: stopping tablet replica
I20260807 01:46:18.668505 24592 raft_consensus.cc:2243] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 3 LEADER]: Raft consensus shutting down.
I20260807 01:46:18.669355 24592 raft_consensus.cc:2272] T a7a55cd1670e40ec94e4269dc15517bb P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.671833 24592 tablet_replica.cc:333] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00: stopping tablet replica
I20260807 01:46:18.672590 24592 raft_consensus.cc:2243] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:18.673105 24592 raft_consensus.cc:2272] T eff0dbafbd3f457ba6c59d0e8bf16705 P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.675091 24592 tablet_replica.cc:333] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00: stopping tablet replica
I20260807 01:46:18.675637 24592 raft_consensus.cc:2243] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:18.676060 24592 raft_consensus.cc:2272] T 06fd48ba8b4b49e2a0f51f1f9b8a67df P b147ae01363d411980d3a99e72915a00 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.677963 24592 tablet_replica.cc:333] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00: stopping tablet replica
I20260807 01:46:18.678516 24592 raft_consensus.cc:2243] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 4 LEADER]: Raft consensus shutting down.
I20260807 01:46:18.679378 24592 raft_consensus.cc:2272] T 75e20a485954424c8b721f0ce6216da2 P b147ae01363d411980d3a99e72915a00 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.681353 24592 tablet_replica.cc:333] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00: stopping tablet replica
I20260807 01:46:18.681830 24592 raft_consensus.cc:2243] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:18.682411 24592 raft_consensus.cc:2272] T 39269a566fe848f481ab0b1a50d48220 P b147ae01363d411980d3a99e72915a00 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.684142 24592 tablet_replica.cc:333] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00: stopping tablet replica
I20260807 01:46:18.684587 24592 raft_consensus.cc:2243] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:18.685010 24592 raft_consensus.cc:2272] T 061b319309704115ac6bc2716e08fd4a P b147ae01363d411980d3a99e72915a00 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.717329 24592 tablet_server.cc:196] TabletServer@127.24.4.3:0 shutdown complete.
I20260807 01:46:18.741483 24592 master.cc:562] Master@127.24.4.62:42161 shutting down...
I20260807 01:46:18.772320 24592 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 01:46:18.773002 24592 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:18.773321 24592 tablet_replica.cc:333] T 00000000000000000000000000000000 P 42fc07fefb5b442bafb484e48a00f8d3: stopping tablet replica
I20260807 01:46:18.795543 24592 master.cc:584] Master@127.24.4.62:42161 shutdown complete.
21df8eb83f204bd595e6ecd8f6405ef8 5
8f9c53697dc142e4af11d5f5e0e167de 4
b147ae01363d411980d3a99e72915a00 1
21df8eb83f204bd595e6ecd8f6405ef8 3
8f9c53697dc142e4af11d5f5e0e167de 4
b147ae01363d411980d3a99e72915a00 3
21df8eb83f204bd595e6ecd8f6405ef8 8
8f9c53697dc142e4af11d5f5e0e167de 1
b147ae01363d411980d3a99e72915a00 1
21df8eb83f204bd595e6ecd8f6405ef8 4
8f9c53697dc142e4af11d5f5e0e167de 3
b147ae01363d411980d3a99e72915a00 3
[ OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (31211 ms)
[ RUN ] LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass
I20260807 01:46:18.847816 24592 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.24.4.62:46391
I20260807 01:46:18.849112 24592 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 01:46:18.856331 26286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 01:46:18.856529 26290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 01:46:18.856563 26294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:46:18.857685 24592 server_base.cc:1061] running on GCE node
I20260807 01:46:18.858819 24592 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 01:46:18.859017 24592 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 01:46:18.859135 24592 hybrid_clock.cc:648] HybridClock initialized: now 1786067178859124 us; error 0 us; skew 500 ppm
I20260807 01:46:18.859601 24592 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 01:46:18.862536 24592 webserver.cc:533] Webserver started at http://127.24.4.62:37335/ using document root <none> and password file <none>
I20260807 01:46:18.863068 24592 fs_manager.cc:362] Metadata directory not provided
I20260807 01:46:18.863241 24592 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 01:46:18.863487 24592 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 01:46:18.864722 24592 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786067112798834-24592-0/minicluster-data/master-0-root/instance:
uuid: "a6c711799ea8410192475e9f80398814"
format_stamp: "Formatted at 2026-08-07 01:46:18 on dist-test-slave-zj8q"
I20260807 01:46:18.869720 24592 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260807 01:46:18.873522 26300 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 01:46:18.874349 24592 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260807 01:46:18.874666 24592 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786067112798834-24592-0/minicluster-data/master-0-root
uuid: "a6c711799ea8410192475e9f80398814"
format_stamp: "Formatted at 2026-08-07 01:46:18 on dist-test-slave-zj8q"
I20260807 01:46:18.874941 24592 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786067112798834-24592-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786067112798834-24592-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786067112798834-24592-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 01:46:18.884743 24592 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 01:46:18.886322 24592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 01:46:18.931463 24592 rpc_server.cc:307] RPC server started. Bound to: 127.24.4.62:46391
I20260807 01:46:18.931541 26386 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.4.62:46391 every 8 connection(s)
I20260807 01:46:18.937000 26390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:18.948635 26390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814: Bootstrap starting.
I20260807 01:46:18.955080 26390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:18.959987 26390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814: No bootstrap required, opened a new log
I20260807 01:46:18.962430 26390 raft_consensus.cc:359] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6c711799ea8410192475e9f80398814" member_type: VOTER }
I20260807 01:46:18.962885 26390 raft_consensus.cc:385] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:18.963141 26390 raft_consensus.cc:740] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6c711799ea8410192475e9f80398814, State: Initialized, Role: FOLLOWER
I20260807 01:46:18.963707 26390 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6c711799ea8410192475e9f80398814" member_type: VOTER }
I20260807 01:46:18.964367 26390 raft_consensus.cc:399] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 01:46:18.964712 26390 raft_consensus.cc:493] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 01:46:18.965008 26390 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:18.969612 26390 raft_consensus.cc:515] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6c711799ea8410192475e9f80398814" member_type: VOTER }
I20260807 01:46:18.970207 26390 leader_election.cc:304] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6c711799ea8410192475e9f80398814; no voters:
I20260807 01:46:18.971644 26390 leader_election.cc:290] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 01:46:18.972182 26396 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:18.973959 26396 raft_consensus.cc:697] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814 [term 1 LEADER]: Becoming Leader. State: Replica: a6c711799ea8410192475e9f80398814, State: Running, Role: LEADER
I20260807 01:46:18.974936 26390 sys_catalog.cc:565] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 01:46:18.974793 26396 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6c711799ea8410192475e9f80398814" member_type: VOTER }
I20260807 01:46:18.981230 26397 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6c711799ea8410192475e9f80398814" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6c711799ea8410192475e9f80398814" member_type: VOTER } }
I20260807 01:46:18.981101 26401 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6c711799ea8410192475e9f80398814. Latest consensus state: current_term: 1 leader_uuid: "a6c711799ea8410192475e9f80398814" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6c711799ea8410192475e9f80398814" member_type: VOTER } }
I20260807 01:46:18.981766 26397 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814 [sys.catalog]: This master's current role is: LEADER
I20260807 01:46:18.981982 26401 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814 [sys.catalog]: This master's current role is: LEADER
I20260807 01:46:18.987074 26411 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260807 01:46:18.993562 26411 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260807 01:46:18.994963 24592 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 01:46:19.005071 26411 catalog_manager.cc:1383] Generated new cluster ID: 012a2fec3efe4449bf8602b9eaf1d9cd
I20260807 01:46:19.005471 26411 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260807 01:46:19.026657 26411 catalog_manager.cc:1406] Generated new certificate authority record
I20260807 01:46:19.028368 26411 catalog_manager.cc:1540] Loading token signing keys...
I20260807 01:46:19.054580 26411 catalog_manager.cc:6070] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814: Generated new TSK 0
I20260807 01:46:19.055280 26411 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260807 01:46:19.061779 24592 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 01:46:19.068918 26432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 01:46:19.069703 26434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 01:46:19.071267 26436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:46:19.073412 24592 server_base.cc:1061] running on GCE node
I20260807 01:46:19.074512 24592 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 01:46:19.074731 24592 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 01:46:19.074892 24592 hybrid_clock.cc:648] HybridClock initialized: now 1786067179074878 us; error 0 us; skew 500 ppm
I20260807 01:46:19.075430 24592 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 01:46:19.078625 24592 webserver.cc:533] Webserver started at http://127.24.4.1:46353/ using document root <none> and password file <none>
I20260807 01:46:19.079180 24592 fs_manager.cc:362] Metadata directory not provided
I20260807 01:46:19.079370 24592 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 01:46:19.079635 24592 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 01:46:19.080849 24592 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786067112798834-24592-0/minicluster-data/ts-0-root/instance:
uuid: "3c8a6986074c4a15912eeca703de1f91"
format_stamp: "Formatted at 2026-08-07 01:46:19 on dist-test-slave-zj8q"
I20260807 01:46:19.085892 24592 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260807 01:46:19.089547 26447 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 01:46:19.090705 24592 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 01:46:19.091142 24592 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786067112798834-24592-0/minicluster-data/ts-0-root
uuid: "3c8a6986074c4a15912eeca703de1f91"
format_stamp: "Formatted at 2026-08-07 01:46:19 on dist-test-slave-zj8q"
I20260807 01:46:19.091495 24592 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786067112798834-24592-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786067112798834-24592-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786067112798834-24592-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 01:46:19.103466 24592 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 01:46:19.104840 24592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 01:46:19.106676 24592 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 01:46:19.110113 24592 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 01:46:19.110404 24592 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 01:46:19.110664 24592 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 01:46:19.110819 24592 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 01:46:19.163805 24592 rpc_server.cc:307] RPC server started. Bound to: 127.24.4.1:40495
I20260807 01:46:19.163861 26557 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.4.1:40495 every 8 connection(s)
I20260807 01:46:19.169032 24592 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 01:46:19.176281 26564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 01:46:19.179461 26565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:46:19.180413 24592 server_base.cc:1061] running on GCE node
W20260807 01:46:19.182109 26567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:46:19.183131 24592 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 01:46:19.183349 24592 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 01:46:19.183518 24592 hybrid_clock.cc:648] HybridClock initialized: now 1786067179183501 us; error 0 us; skew 500 ppm
I20260807 01:46:19.184207 24592 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 01:46:19.186365 26558 heartbeater.cc:344] Connected to a master server at 127.24.4.62:46391
I20260807 01:46:19.186735 26558 heartbeater.cc:461] Registering TS with master...
I20260807 01:46:19.187247 24592 webserver.cc:533] Webserver started at http://127.24.4.2:42215/ using document root <none> and password file <none>
I20260807 01:46:19.187583 26558 heartbeater.cc:507] Master 127.24.4.62:46391 requested a full tablet report, sending...
I20260807 01:46:19.187788 24592 fs_manager.cc:362] Metadata directory not provided
I20260807 01:46:19.188057 24592 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 01:46:19.188320 24592 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 01:46:19.189651 24592 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786067112798834-24592-0/minicluster-data/ts-1-root/instance:
uuid: "5fe3de95e0e3415badd3ac197021f761"
format_stamp: "Formatted at 2026-08-07 01:46:19 on dist-test-slave-zj8q"
I20260807 01:46:19.190073 26324 ts_manager.cc:194] Registered new tserver with Master: 3c8a6986074c4a15912eeca703de1f91 (127.24.4.1:40495)
I20260807 01:46:19.192492 26324 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55998
I20260807 01:46:19.195794 24592 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260807 01:46:19.199927 26575 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 01:46:19.200855 24592 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 01:46:19.201211 24592 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786067112798834-24592-0/minicluster-data/ts-1-root
uuid: "5fe3de95e0e3415badd3ac197021f761"
format_stamp: "Formatted at 2026-08-07 01:46:19 on dist-test-slave-zj8q"
I20260807 01:46:19.201515 24592 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786067112798834-24592-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786067112798834-24592-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786067112798834-24592-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 01:46:19.224886 24592 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 01:46:19.226518 24592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 01:46:19.228725 24592 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 01:46:19.232476 24592 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 01:46:19.232860 24592 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 01:46:19.233182 24592 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 01:46:19.233353 24592 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 01:46:19.294296 24592 rpc_server.cc:307] RPC server started. Bound to: 127.24.4.2:44813
I20260807 01:46:19.294405 26674 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.4.2:44813 every 8 connection(s)
I20260807 01:46:19.299293 24592 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 01:46:19.307816 26681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 01:46:19.310024 26682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:46:19.312723 24592 server_base.cc:1061] running on GCE node
W20260807 01:46:19.312754 26685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:46:19.314087 24592 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260807 01:46:19.314091 26675 heartbeater.cc:344] Connected to a master server at 127.24.4.62:46391
W20260807 01:46:19.314558 24592 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 01:46:19.314741 26675 heartbeater.cc:461] Registering TS with master...
I20260807 01:46:19.314780 24592 hybrid_clock.cc:648] HybridClock initialized: now 1786067179314761 us; error 0 us; skew 500 ppm
I20260807 01:46:19.315701 24592 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 01:46:19.315842 26675 heartbeater.cc:507] Master 127.24.4.62:46391 requested a full tablet report, sending...
I20260807 01:46:19.318866 26324 ts_manager.cc:194] Registered new tserver with Master: 5fe3de95e0e3415badd3ac197021f761 (127.24.4.2:44813)
I20260807 01:46:19.319067 24592 webserver.cc:533] Webserver started at http://127.24.4.3:43723/ using document root <none> and password file <none>
I20260807 01:46:19.319643 24592 fs_manager.cc:362] Metadata directory not provided
I20260807 01:46:19.319844 24592 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 01:46:19.320107 24592 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 01:46:19.320503 26324 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56008
I20260807 01:46:19.321465 24592 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786067112798834-24592-0/minicluster-data/ts-2-root/instance:
uuid: "03e20eec798943849f25342a646c51ca"
format_stamp: "Formatted at 2026-08-07 01:46:19 on dist-test-slave-zj8q"
I20260807 01:46:19.327044 24592 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260807 01:46:19.330744 26691 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 01:46:19.331615 24592 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 01:46:19.331923 24592 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786067112798834-24592-0/minicluster-data/ts-2-root
uuid: "03e20eec798943849f25342a646c51ca"
format_stamp: "Formatted at 2026-08-07 01:46:19 on dist-test-slave-zj8q"
I20260807 01:46:19.332233 24592 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786067112798834-24592-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786067112798834-24592-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786067112798834-24592-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 01:46:19.349512 24592 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 01:46:19.351022 24592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 01:46:19.352759 24592 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 01:46:19.356237 24592 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 01:46:19.356495 24592 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 01:46:19.356756 24592 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 01:46:19.356913 24592 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 01:46:19.415285 24592 rpc_server.cc:307] RPC server started. Bound to: 127.24.4.3:39659
I20260807 01:46:19.415478 26786 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.4.3:39659 every 8 connection(s)
I20260807 01:46:19.432158 26787 heartbeater.cc:344] Connected to a master server at 127.24.4.62:46391
I20260807 01:46:19.432622 26787 heartbeater.cc:461] Registering TS with master...
I20260807 01:46:19.433503 26787 heartbeater.cc:507] Master 127.24.4.62:46391 requested a full tablet report, sending...
I20260807 01:46:19.436223 26324 ts_manager.cc:194] Registered new tserver with Master: 03e20eec798943849f25342a646c51ca (127.24.4.3:39659)
I20260807 01:46:19.436959 24592 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01813717s
I20260807 01:46:19.438014 26324 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56024
I20260807 01:46:19.471773 26324 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56028:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000UUU\025\004\001\000\252\252\252*\004\001\000\377\377\377?\004\001\000TUUU\004\001\000\251\252\252j""\004\001\000UUU\025\004\001\000\252\252\252*\004\001\000\377\377\377?\004\001\000TUUU\004\001\000\251\252\252j"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 01:46:19.474853 26324 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 01:46:19.551681 26732 tablet_service.cc:1511] Processing CreateTablet for tablet 0e9fce5dee804aa7ab8d61ee6528d05a (DEFAULT_TABLE table=test-workload [id=165b3de7ebab4cbbb8db09f46b9fe527]), partition=RANGE (key) PARTITION VALUES < 357913941
I20260807 01:46:19.552529 26730 tablet_service.cc:1511] Processing CreateTablet for tablet f07aec3c0cac4b28ad53b3ffcb67734b (DEFAULT_TABLE table=test-workload [id=165b3de7ebab4cbbb8db09f46b9fe527]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1073741823
I20260807 01:46:19.553604 26732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e9fce5dee804aa7ab8d61ee6528d05a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:19.553725 26619 tablet_service.cc:1511] Processing CreateTablet for tablet 0e9fce5dee804aa7ab8d61ee6528d05a (DEFAULT_TABLE table=test-workload [id=165b3de7ebab4cbbb8db09f46b9fe527]), partition=RANGE (key) PARTITION VALUES < 357913941
I20260807 01:46:19.554571 26730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f07aec3c0cac4b28ad53b3ffcb67734b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:19.555332 26619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e9fce5dee804aa7ab8d61ee6528d05a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:19.556211 26617 tablet_service.cc:1511] Processing CreateTablet for tablet 1cb9ab528b474779946bebdd16dc46ab (DEFAULT_TABLE table=test-workload [id=165b3de7ebab4cbbb8db09f46b9fe527]), partition=RANGE (key) PARTITION 357913941 <= VALUES < 715827882
I20260807 01:46:19.552565 26731 tablet_service.cc:1511] Processing CreateTablet for tablet 1cb9ab528b474779946bebdd16dc46ab (DEFAULT_TABLE table=test-workload [id=165b3de7ebab4cbbb8db09f46b9fe527]), partition=RANGE (key) PARTITION 357913941 <= VALUES < 715827882
I20260807 01:46:19.559204 26731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cb9ab528b474779946bebdd16dc46ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:19.559360 26616 tablet_service.cc:1511] Processing CreateTablet for tablet f07aec3c0cac4b28ad53b3ffcb67734b (DEFAULT_TABLE table=test-workload [id=165b3de7ebab4cbbb8db09f46b9fe527]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1073741823
I20260807 01:46:19.557695 26617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cb9ab528b474779946bebdd16dc46ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:19.564531 26614 tablet_service.cc:1511] Processing CreateTablet for tablet 3aff9336a59748629accf669219a4490 (DEFAULT_TABLE table=test-workload [id=165b3de7ebab4cbbb8db09f46b9fe527]), partition=RANGE (key) PARTITION 1073741823 <= VALUES < 1431655764
I20260807 01:46:19.570575 26614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aff9336a59748629accf669219a4490. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:19.574337 26612 tablet_service.cc:1511] Processing CreateTablet for tablet 215351a405d34d1eab01fa98b8f066d6 (DEFAULT_TABLE table=test-workload [id=165b3de7ebab4cbbb8db09f46b9fe527]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1789569705
I20260807 01:46:19.575887 26612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 215351a405d34d1eab01fa98b8f066d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:19.576368 26611 tablet_service.cc:1511] Processing CreateTablet for tablet 9a5fdee17bfc441f9a2ef24cc5b6f2e7 (DEFAULT_TABLE table=test-workload [id=165b3de7ebab4cbbb8db09f46b9fe527]), partition=RANGE (key) PARTITION 1789569705 <= VALUES
I20260807 01:46:19.577870 26611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a5fdee17bfc441f9a2ef24cc5b6f2e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:19.582538 26616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f07aec3c0cac4b28ad53b3ffcb67734b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:19.587702 26728 tablet_service.cc:1511] Processing CreateTablet for tablet 3aff9336a59748629accf669219a4490 (DEFAULT_TABLE table=test-workload [id=165b3de7ebab4cbbb8db09f46b9fe527]), partition=RANGE (key) PARTITION 1073741823 <= VALUES < 1431655764
I20260807 01:46:19.589326 26728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aff9336a59748629accf669219a4490. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:19.592065 26726 tablet_service.cc:1511] Processing CreateTablet for tablet 215351a405d34d1eab01fa98b8f066d6 (DEFAULT_TABLE table=test-workload [id=165b3de7ebab4cbbb8db09f46b9fe527]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1789569705
I20260807 01:46:19.593547 26726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 215351a405d34d1eab01fa98b8f066d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:19.596110 26723 tablet_service.cc:1511] Processing CreateTablet for tablet 9a5fdee17bfc441f9a2ef24cc5b6f2e7 (DEFAULT_TABLE table=test-workload [id=165b3de7ebab4cbbb8db09f46b9fe527]), partition=RANGE (key) PARTITION 1789569705 <= VALUES
I20260807 01:46:19.596894 26816 tablet_bootstrap.cc:492] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761: Bootstrap starting.
I20260807 01:46:19.597628 26723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a5fdee17bfc441f9a2ef24cc5b6f2e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:19.606324 26816 tablet_bootstrap.cc:654] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:19.653036 26816 tablet_bootstrap.cc:492] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761: No bootstrap required, opened a new log
I20260807 01:46:19.653766 26816 ts_tablet_manager.cc:1403] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761: Time spent bootstrapping tablet: real 0.057s user 0.016s sys 0.004s
I20260807 01:46:19.655051 26495 tablet_service.cc:1511] Processing CreateTablet for tablet 1cb9ab528b474779946bebdd16dc46ab (DEFAULT_TABLE table=test-workload [id=165b3de7ebab4cbbb8db09f46b9fe527]), partition=RANGE (key) PARTITION 357913941 <= VALUES < 715827882
I20260807 01:46:19.656009 26491 tablet_service.cc:1511] Processing CreateTablet for tablet 215351a405d34d1eab01fa98b8f066d6 (DEFAULT_TABLE table=test-workload [id=165b3de7ebab4cbbb8db09f46b9fe527]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1789569705
I20260807 01:46:19.656749 26495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cb9ab528b474779946bebdd16dc46ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:19.657665 26491 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 215351a405d34d1eab01fa98b8f066d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:19.658324 26816 raft_consensus.cc:359] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } }
I20260807 01:46:19.659135 26816 raft_consensus.cc:385] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:19.659452 26816 raft_consensus.cc:740] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fe3de95e0e3415badd3ac197021f761, State: Initialized, Role: FOLLOWER
I20260807 01:46:19.660287 26816 consensus_queue.cc:260] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } }
I20260807 01:46:19.660738 26489 tablet_service.cc:1511] Processing CreateTablet for tablet 9a5fdee17bfc441f9a2ef24cc5b6f2e7 (DEFAULT_TABLE table=test-workload [id=165b3de7ebab4cbbb8db09f46b9fe527]), partition=RANGE (key) PARTITION 1789569705 <= VALUES
I20260807 01:46:19.667498 26494 tablet_service.cc:1511] Processing CreateTablet for tablet f07aec3c0cac4b28ad53b3ffcb67734b (DEFAULT_TABLE table=test-workload [id=165b3de7ebab4cbbb8db09f46b9fe527]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1073741823
I20260807 01:46:19.659453 26493 tablet_service.cc:1511] Processing CreateTablet for tablet 3aff9336a59748629accf669219a4490 (DEFAULT_TABLE table=test-workload [id=165b3de7ebab4cbbb8db09f46b9fe527]), partition=RANGE (key) PARTITION 1073741823 <= VALUES < 1431655764
I20260807 01:46:19.656435 26496 tablet_service.cc:1511] Processing CreateTablet for tablet 0e9fce5dee804aa7ab8d61ee6528d05a (DEFAULT_TABLE table=test-workload [id=165b3de7ebab4cbbb8db09f46b9fe527]), partition=RANGE (key) PARTITION VALUES < 357913941
I20260807 01:46:19.673236 26489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a5fdee17bfc441f9a2ef24cc5b6f2e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:19.674711 26496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e9fce5dee804aa7ab8d61ee6528d05a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:19.679770 26493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aff9336a59748629accf669219a4490. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:19.681478 26494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f07aec3c0cac4b28ad53b3ffcb67734b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:19.708214 26675 heartbeater.cc:499] Master 127.24.4.62:46391 was elected leader, sending a full tablet report...
I20260807 01:46:19.709187 26816 ts_tablet_manager.cc:1434] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761: Time spent starting tablet: real 0.055s user 0.013s sys 0.030s
I20260807 01:46:19.711478 26816 tablet_bootstrap.cc:492] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 5fe3de95e0e3415badd3ac197021f761: Bootstrap starting.
I20260807 01:46:19.719085 26816 tablet_bootstrap.cc:654] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 5fe3de95e0e3415badd3ac197021f761: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:19.732550 26814 tablet_bootstrap.cc:492] T 1cb9ab528b474779946bebdd16dc46ab P 03e20eec798943849f25342a646c51ca: Bootstrap starting.
I20260807 01:46:19.771840 26820 tablet_bootstrap.cc:492] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91: Bootstrap starting.
I20260807 01:46:19.772125 26814 tablet_bootstrap.cc:654] T 1cb9ab528b474779946bebdd16dc46ab P 03e20eec798943849f25342a646c51ca: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:19.783871 26820 tablet_bootstrap.cc:654] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:19.786680 26816 tablet_bootstrap.cc:492] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 5fe3de95e0e3415badd3ac197021f761: No bootstrap required, opened a new log
I20260807 01:46:19.786417 26814 tablet_bootstrap.cc:492] T 1cb9ab528b474779946bebdd16dc46ab P 03e20eec798943849f25342a646c51ca: No bootstrap required, opened a new log
I20260807 01:46:19.787443 26816 ts_tablet_manager.cc:1403] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 5fe3de95e0e3415badd3ac197021f761: Time spent bootstrapping tablet: real 0.076s user 0.019s sys 0.040s
I20260807 01:46:19.787488 26814 ts_tablet_manager.cc:1403] T 1cb9ab528b474779946bebdd16dc46ab P 03e20eec798943849f25342a646c51ca: Time spent bootstrapping tablet: real 0.055s user 0.013s sys 0.005s
I20260807 01:46:19.791692 26816 raft_consensus.cc:359] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.792488 26816 raft_consensus.cc:385] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:19.792135 26814 raft_consensus.cc:359] T 1cb9ab528b474779946bebdd16dc46ab P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.792809 26816 raft_consensus.cc:740] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fe3de95e0e3415badd3ac197021f761, State: Initialized, Role: FOLLOWER
I20260807 01:46:19.793027 26814 raft_consensus.cc:385] T 1cb9ab528b474779946bebdd16dc46ab P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:19.793109 26820 tablet_bootstrap.cc:492] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91: No bootstrap required, opened a new log
I20260807 01:46:19.793593 26814 raft_consensus.cc:740] T 1cb9ab528b474779946bebdd16dc46ab P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03e20eec798943849f25342a646c51ca, State: Initialized, Role: FOLLOWER
I20260807 01:46:19.793763 26820 ts_tablet_manager.cc:1403] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91: Time spent bootstrapping tablet: real 0.023s user 0.019s sys 0.000s
I20260807 01:46:19.793641 26816 consensus_queue.cc:260] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 5fe3de95e0e3415badd3ac197021f761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.794629 26814 consensus_queue.cc:260] T 1cb9ab528b474779946bebdd16dc46ab P 03e20eec798943849f25342a646c51ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.796006 26816 ts_tablet_manager.cc:1434] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 5fe3de95e0e3415badd3ac197021f761: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260807 01:46:19.797084 26816 tablet_bootstrap.cc:492] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761: Bootstrap starting.
I20260807 01:46:19.796995 26820 raft_consensus.cc:359] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.797974 26820 raft_consensus.cc:385] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:19.798321 26820 raft_consensus.cc:740] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c8a6986074c4a15912eeca703de1f91, State: Initialized, Role: FOLLOWER
W20260807 01:46:19.798755 26677 tablet.cc:2406] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 01:46:19.799544 26820 consensus_queue.cc:260] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.804204 26814 ts_tablet_manager.cc:1434] T 1cb9ab528b474779946bebdd16dc46ab P 03e20eec798943849f25342a646c51ca: Time spent starting tablet: real 0.016s user 0.006s sys 0.007s
I20260807 01:46:19.804387 26787 heartbeater.cc:499] Master 127.24.4.62:46391 was elected leader, sending a full tablet report...
I20260807 01:46:19.804986 26816 tablet_bootstrap.cc:654] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:19.805377 26814 tablet_bootstrap.cc:492] T f07aec3c0cac4b28ad53b3ffcb67734b P 03e20eec798943849f25342a646c51ca: Bootstrap starting.
I20260807 01:46:19.812541 26814 tablet_bootstrap.cc:654] T f07aec3c0cac4b28ad53b3ffcb67734b P 03e20eec798943849f25342a646c51ca: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:19.817348 26816 tablet_bootstrap.cc:492] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761: No bootstrap required, opened a new log
I20260807 01:46:19.818032 26816 ts_tablet_manager.cc:1403] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.016s
I20260807 01:46:19.818131 26820 ts_tablet_manager.cc:1434] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91: Time spent starting tablet: real 0.024s user 0.006s sys 0.004s
I20260807 01:46:19.818981 26558 heartbeater.cc:499] Master 127.24.4.62:46391 was elected leader, sending a full tablet report...
I20260807 01:46:19.821617 26816 raft_consensus.cc:359] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.822623 26816 raft_consensus.cc:385] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:19.823004 26816 raft_consensus.cc:740] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fe3de95e0e3415badd3ac197021f761, State: Initialized, Role: FOLLOWER
I20260807 01:46:19.823962 26816 consensus_queue.cc:260] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.827279 26820 tablet_bootstrap.cc:492] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91: Bootstrap starting.
I20260807 01:46:19.829066 26814 tablet_bootstrap.cc:492] T f07aec3c0cac4b28ad53b3ffcb67734b P 03e20eec798943849f25342a646c51ca: No bootstrap required, opened a new log
I20260807 01:46:19.829663 26814 ts_tablet_manager.cc:1403] T f07aec3c0cac4b28ad53b3ffcb67734b P 03e20eec798943849f25342a646c51ca: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.015s
I20260807 01:46:19.831513 26816 ts_tablet_manager.cc:1434] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20260807 01:46:19.833036 26816 tablet_bootstrap.cc:492] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761: Bootstrap starting.
I20260807 01:46:19.832939 26814 raft_consensus.cc:359] T f07aec3c0cac4b28ad53b3ffcb67734b P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.834021 26814 raft_consensus.cc:385] T f07aec3c0cac4b28ad53b3ffcb67734b P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:19.834381 26814 raft_consensus.cc:740] T f07aec3c0cac4b28ad53b3ffcb67734b P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03e20eec798943849f25342a646c51ca, State: Initialized, Role: FOLLOWER
I20260807 01:46:19.834771 26820 tablet_bootstrap.cc:654] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:19.835357 26814 consensus_queue.cc:260] T f07aec3c0cac4b28ad53b3ffcb67734b P 03e20eec798943849f25342a646c51ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.837991 26814 ts_tablet_manager.cc:1434] T f07aec3c0cac4b28ad53b3ffcb67734b P 03e20eec798943849f25342a646c51ca: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 01:46:19.839175 26814 tablet_bootstrap.cc:492] T 0e9fce5dee804aa7ab8d61ee6528d05a P 03e20eec798943849f25342a646c51ca: Bootstrap starting.
I20260807 01:46:19.841239 26816 tablet_bootstrap.cc:654] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:19.845917 26814 tablet_bootstrap.cc:654] T 0e9fce5dee804aa7ab8d61ee6528d05a P 03e20eec798943849f25342a646c51ca: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:19.851591 26816 tablet_bootstrap.cc:492] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761: No bootstrap required, opened a new log
I20260807 01:46:19.852121 26816 ts_tablet_manager.cc:1403] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761: Time spent bootstrapping tablet: real 0.019s user 0.013s sys 0.003s
I20260807 01:46:19.853215 26814 tablet_bootstrap.cc:492] T 0e9fce5dee804aa7ab8d61ee6528d05a P 03e20eec798943849f25342a646c51ca: No bootstrap required, opened a new log
I20260807 01:46:19.853700 26814 ts_tablet_manager.cc:1403] T 0e9fce5dee804aa7ab8d61ee6528d05a P 03e20eec798943849f25342a646c51ca: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.002s
I20260807 01:46:19.852834 26820 tablet_bootstrap.cc:492] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91: No bootstrap required, opened a new log
I20260807 01:46:19.855262 26820 ts_tablet_manager.cc:1403] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91: Time spent bootstrapping tablet: real 0.028s user 0.007s sys 0.009s
I20260807 01:46:19.855060 26816 raft_consensus.cc:359] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.855852 26816 raft_consensus.cc:385] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:19.856175 26816 raft_consensus.cc:740] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fe3de95e0e3415badd3ac197021f761, State: Initialized, Role: FOLLOWER
I20260807 01:46:19.856580 26814 raft_consensus.cc:359] T 0e9fce5dee804aa7ab8d61ee6528d05a P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } }
I20260807 01:46:19.857288 26814 raft_consensus.cc:385] T 0e9fce5dee804aa7ab8d61ee6528d05a P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:19.856937 26816 consensus_queue.cc:260] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.857601 26814 raft_consensus.cc:740] T 0e9fce5dee804aa7ab8d61ee6528d05a P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03e20eec798943849f25342a646c51ca, State: Initialized, Role: FOLLOWER
I20260807 01:46:19.857703 26820 raft_consensus.cc:359] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.858541 26820 raft_consensus.cc:385] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:19.858386 26814 consensus_queue.cc:260] T 0e9fce5dee804aa7ab8d61ee6528d05a P 03e20eec798943849f25342a646c51ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } }
I20260807 01:46:19.858862 26820 raft_consensus.cc:740] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c8a6986074c4a15912eeca703de1f91, State: Initialized, Role: FOLLOWER
I20260807 01:46:19.859234 26816 ts_tablet_manager.cc:1434] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260807 01:46:19.860276 26816 tablet_bootstrap.cc:492] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761: Bootstrap starting.
I20260807 01:46:19.860662 26814 ts_tablet_manager.cc:1434] T 0e9fce5dee804aa7ab8d61ee6528d05a P 03e20eec798943849f25342a646c51ca: Time spent starting tablet: real 0.007s user 0.002s sys 0.005s
I20260807 01:46:19.861621 26814 tablet_bootstrap.cc:492] T 215351a405d34d1eab01fa98b8f066d6 P 03e20eec798943849f25342a646c51ca: Bootstrap starting.
I20260807 01:46:19.864106 26820 consensus_queue.cc:260] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.866452 26820 ts_tablet_manager.cc:1434] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91: Time spent starting tablet: real 0.011s user 0.005s sys 0.001s
I20260807 01:46:19.867098 26816 tablet_bootstrap.cc:654] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:19.867424 26820 tablet_bootstrap.cc:492] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91: Bootstrap starting.
I20260807 01:46:19.868251 26814 tablet_bootstrap.cc:654] T 215351a405d34d1eab01fa98b8f066d6 P 03e20eec798943849f25342a646c51ca: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:19.874361 26820 tablet_bootstrap.cc:654] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:19.892283 26816 tablet_bootstrap.cc:492] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761: No bootstrap required, opened a new log
I20260807 01:46:19.893508 26816 ts_tablet_manager.cc:1403] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761: Time spent bootstrapping tablet: real 0.033s user 0.020s sys 0.009s
I20260807 01:46:19.893977 26820 tablet_bootstrap.cc:492] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91: No bootstrap required, opened a new log
I20260807 01:46:19.894606 26820 ts_tablet_manager.cc:1403] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91: Time spent bootstrapping tablet: real 0.027s user 0.005s sys 0.011s
I20260807 01:46:19.894794 26814 tablet_bootstrap.cc:492] T 215351a405d34d1eab01fa98b8f066d6 P 03e20eec798943849f25342a646c51ca: No bootstrap required, opened a new log
I20260807 01:46:19.895243 26814 ts_tablet_manager.cc:1403] T 215351a405d34d1eab01fa98b8f066d6 P 03e20eec798943849f25342a646c51ca: Time spent bootstrapping tablet: real 0.034s user 0.025s sys 0.004s
I20260807 01:46:19.896581 26816 raft_consensus.cc:359] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.897362 26816 raft_consensus.cc:385] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:19.897668 26816 raft_consensus.cc:740] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fe3de95e0e3415badd3ac197021f761, State: Initialized, Role: FOLLOWER
I20260807 01:46:19.897698 26814 raft_consensus.cc:359] T 215351a405d34d1eab01fa98b8f066d6 P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.897603 26820 raft_consensus.cc:359] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.898514 26814 raft_consensus.cc:385] T 215351a405d34d1eab01fa98b8f066d6 P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:19.898573 26820 raft_consensus.cc:385] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:19.898948 26814 raft_consensus.cc:740] T 215351a405d34d1eab01fa98b8f066d6 P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03e20eec798943849f25342a646c51ca, State: Initialized, Role: FOLLOWER
I20260807 01:46:19.899037 26820 raft_consensus.cc:740] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c8a6986074c4a15912eeca703de1f91, State: Initialized, Role: FOLLOWER
I20260807 01:46:19.898677 26816 consensus_queue.cc:260] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.899832 26814 consensus_queue.cc:260] T 215351a405d34d1eab01fa98b8f066d6 P 03e20eec798943849f25342a646c51ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.899919 26820 consensus_queue.cc:260] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.901099 26816 ts_tablet_manager.cc:1434] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 01:46:19.902117 26816 tablet_bootstrap.cc:492] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761: Bootstrap starting.
I20260807 01:46:19.904038 26820 ts_tablet_manager.cc:1434] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91: Time spent starting tablet: real 0.009s user 0.006s sys 0.001s
I20260807 01:46:19.905133 26820 tablet_bootstrap.cc:492] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91: Bootstrap starting.
I20260807 01:46:19.904084 26814 ts_tablet_manager.cc:1434] T 215351a405d34d1eab01fa98b8f066d6 P 03e20eec798943849f25342a646c51ca: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260807 01:46:19.907297 26814 tablet_bootstrap.cc:492] T 3aff9336a59748629accf669219a4490 P 03e20eec798943849f25342a646c51ca: Bootstrap starting.
I20260807 01:46:19.908843 26816 tablet_bootstrap.cc:654] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:19.913139 26820 tablet_bootstrap.cc:654] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:19.913694 26814 tablet_bootstrap.cc:654] T 3aff9336a59748629accf669219a4490 P 03e20eec798943849f25342a646c51ca: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:19.939978 26816 tablet_bootstrap.cc:492] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761: No bootstrap required, opened a new log
I20260807 01:46:19.940551 26816 ts_tablet_manager.cc:1403] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761: Time spent bootstrapping tablet: real 0.039s user 0.016s sys 0.020s
I20260807 01:46:19.942237 26814 tablet_bootstrap.cc:492] T 3aff9336a59748629accf669219a4490 P 03e20eec798943849f25342a646c51ca: No bootstrap required, opened a new log
I20260807 01:46:19.942304 26820 tablet_bootstrap.cc:492] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91: No bootstrap required, opened a new log
I20260807 01:46:19.943040 26814 ts_tablet_manager.cc:1403] T 3aff9336a59748629accf669219a4490 P 03e20eec798943849f25342a646c51ca: Time spent bootstrapping tablet: real 0.036s user 0.006s sys 0.028s
I20260807 01:46:19.943070 26820 ts_tablet_manager.cc:1403] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91: Time spent bootstrapping tablet: real 0.038s user 0.014s sys 0.018s
I20260807 01:46:19.943215 26818 raft_consensus.cc:493] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:19.943805 26818 raft_consensus.cc:515] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.946686 26818 leader_election.cc:290] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c8a6986074c4a15912eeca703de1f91 (127.24.4.1:40495), 03e20eec798943849f25342a646c51ca (127.24.4.3:39659)
I20260807 01:46:19.946484 26820 raft_consensus.cc:359] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.947286 26820 raft_consensus.cc:385] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:19.946821 26814 raft_consensus.cc:359] T 3aff9336a59748629accf669219a4490 P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.947685 26820 raft_consensus.cc:740] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c8a6986074c4a15912eeca703de1f91, State: Initialized, Role: FOLLOWER
I20260807 01:46:19.947242 26818 raft_consensus.cc:493] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:19.948091 26814 raft_consensus.cc:385] T 3aff9336a59748629accf669219a4490 P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:19.951565 26814 raft_consensus.cc:740] T 3aff9336a59748629accf669219a4490 P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03e20eec798943849f25342a646c51ca, State: Initialized, Role: FOLLOWER
I20260807 01:46:19.948746 26818 raft_consensus.cc:515] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.952317 26816 raft_consensus.cc:359] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.953114 26816 raft_consensus.cc:385] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:19.952440 26814 consensus_queue.cc:260] T 3aff9336a59748629accf669219a4490 P 03e20eec798943849f25342a646c51ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.953474 26816 raft_consensus.cc:740] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fe3de95e0e3415badd3ac197021f761, State: Initialized, Role: FOLLOWER
I20260807 01:46:19.954505 26818 leader_election.cc:290] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c8a6986074c4a15912eeca703de1f91 (127.24.4.1:40495), 03e20eec798943849f25342a646c51ca (127.24.4.3:39659)
I20260807 01:46:19.954362 26816 consensus_queue.cc:260] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.955722 26820 consensus_queue.cc:260] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.955250 26814 ts_tablet_manager.cc:1434] T 3aff9336a59748629accf669219a4490 P 03e20eec798943849f25342a646c51ca: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260807 01:46:19.958891 26814 tablet_bootstrap.cc:492] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca: Bootstrap starting.
I20260807 01:46:19.958988 26820 ts_tablet_manager.cc:1434] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91: Time spent starting tablet: real 0.015s user 0.005s sys 0.001s
I20260807 01:46:19.961213 26820 tablet_bootstrap.cc:492] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91: Bootstrap starting.
I20260807 01:46:19.966657 26816 ts_tablet_manager.cc:1434] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761: Time spent starting tablet: real 0.026s user 0.010s sys 0.000s
I20260807 01:46:19.968650 26820 tablet_bootstrap.cc:654] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:19.974032 26814 tablet_bootstrap.cc:654] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:19.985009 26516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "215351a405d34d1eab01fa98b8f066d6" candidate_uuid: "5fe3de95e0e3415badd3ac197021f761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c8a6986074c4a15912eeca703de1f91" is_pre_election: true
I20260807 01:46:19.985702 26745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "215351a405d34d1eab01fa98b8f066d6" candidate_uuid: "5fe3de95e0e3415badd3ac197021f761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03e20eec798943849f25342a646c51ca" is_pre_election: true
I20260807 01:46:19.986380 26515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f07aec3c0cac4b28ad53b3ffcb67734b" candidate_uuid: "5fe3de95e0e3415badd3ac197021f761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c8a6986074c4a15912eeca703de1f91" is_pre_election: true
I20260807 01:46:19.986459 26745 raft_consensus.cc:2468] T 215351a405d34d1eab01fa98b8f066d6 P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fe3de95e0e3415badd3ac197021f761 in term 0.
I20260807 01:46:19.987020 26515 raft_consensus.cc:2468] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fe3de95e0e3415badd3ac197021f761 in term 0.
I20260807 01:46:19.987499 26814 tablet_bootstrap.cc:492] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca: No bootstrap required, opened a new log
I20260807 01:46:19.987996 26814 ts_tablet_manager.cc:1403] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca: Time spent bootstrapping tablet: real 0.029s user 0.012s sys 0.006s
I20260807 01:46:19.985821 26516 raft_consensus.cc:2468] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fe3de95e0e3415badd3ac197021f761 in term 0.
W20260807 01:46:19.988451 26418 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 01:46:19.988556 26745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f07aec3c0cac4b28ad53b3ffcb67734b" candidate_uuid: "5fe3de95e0e3415badd3ac197021f761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03e20eec798943849f25342a646c51ca" is_pre_election: true
I20260807 01:46:19.989176 26745 raft_consensus.cc:2468] T f07aec3c0cac4b28ad53b3ffcb67734b P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fe3de95e0e3415badd3ac197021f761 in term 0.
I20260807 01:46:19.989429 26578 leader_election.cc:304] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c8a6986074c4a15912eeca703de1f91, 5fe3de95e0e3415badd3ac197021f761; no voters:
I20260807 01:46:19.990008 26577 leader_election.cc:304] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03e20eec798943849f25342a646c51ca, 5fe3de95e0e3415badd3ac197021f761; no voters:
I20260807 01:46:19.990141 26818 raft_consensus.cc:2804] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:19.990598 26818 raft_consensus.cc:493] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:19.990935 26818 raft_consensus.cc:3060] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:19.991144 26814 raft_consensus.cc:359] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.991827 26814 raft_consensus.cc:385] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:19.992108 26814 raft_consensus.cc:740] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03e20eec798943849f25342a646c51ca, State: Initialized, Role: FOLLOWER
I20260807 01:46:19.993021 26814 consensus_queue.cc:260] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:19.994803 26820 tablet_bootstrap.cc:492] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91: No bootstrap required, opened a new log
I20260807 01:46:19.995325 26820 ts_tablet_manager.cc:1403] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91: Time spent bootstrapping tablet: real 0.034s user 0.011s sys 0.006s
I20260807 01:46:19.995682 26814 ts_tablet_manager.cc:1434] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 01:46:19.997920 26820 raft_consensus.cc:359] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } }
I20260807 01:46:19.998679 26820 raft_consensus.cc:385] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:19.999042 26820 raft_consensus.cc:740] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c8a6986074c4a15912eeca703de1f91, State: Initialized, Role: FOLLOWER
I20260807 01:46:19.999776 26820 consensus_queue.cc:260] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } }
I20260807 01:46:20.000227 26852 raft_consensus.cc:2804] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:20.000696 26852 raft_consensus.cc:493] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:19.998055 26818 raft_consensus.cc:515] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.001163 26852 raft_consensus.cc:3060] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:20.003698 26820 ts_tablet_manager.cc:1434] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91: Time spent starting tablet: real 0.008s user 0.004s sys 0.003s
I20260807 01:46:20.003561 26516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f07aec3c0cac4b28ad53b3ffcb67734b" candidate_uuid: "5fe3de95e0e3415badd3ac197021f761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c8a6986074c4a15912eeca703de1f91"
I20260807 01:46:20.004323 26516 raft_consensus.cc:3060] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:20.004979 26820 tablet_bootstrap.cc:492] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91: Bootstrap starting.
I20260807 01:46:20.008908 26852 raft_consensus.cc:515] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.010223 26820 tablet_bootstrap.cc:654] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:20.011178 26852 leader_election.cc:290] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [CANDIDATE]: Term 1 election: Requested vote from peers 3c8a6986074c4a15912eeca703de1f91 (127.24.4.1:40495), 03e20eec798943849f25342a646c51ca (127.24.4.3:39659)
I20260807 01:46:20.011734 26852 raft_consensus.cc:493] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:20.012177 26852 raft_consensus.cc:515] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.012696 26516 raft_consensus.cc:2468] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fe3de95e0e3415badd3ac197021f761 in term 1.
I20260807 01:46:20.014199 26852 leader_election.cc:290] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c8a6986074c4a15912eeca703de1f91 (127.24.4.1:40495), 03e20eec798943849f25342a646c51ca (127.24.4.3:39659)
I20260807 01:46:20.014901 26745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "215351a405d34d1eab01fa98b8f066d6" candidate_uuid: "5fe3de95e0e3415badd3ac197021f761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03e20eec798943849f25342a646c51ca"
I20260807 01:46:20.015468 26745 raft_consensus.cc:3060] T 215351a405d34d1eab01fa98b8f066d6 P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:20.027189 26818 leader_election.cc:290] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [CANDIDATE]: Term 1 election: Requested vote from peers 3c8a6986074c4a15912eeca703de1f91 (127.24.4.1:40495), 03e20eec798943849f25342a646c51ca (127.24.4.3:39659)
I20260807 01:46:20.029520 26744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aff9336a59748629accf669219a4490" candidate_uuid: "5fe3de95e0e3415badd3ac197021f761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03e20eec798943849f25342a646c51ca" is_pre_election: true
I20260807 01:46:20.029665 26743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f07aec3c0cac4b28ad53b3ffcb67734b" candidate_uuid: "5fe3de95e0e3415badd3ac197021f761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03e20eec798943849f25342a646c51ca"
I20260807 01:46:20.030268 26744 raft_consensus.cc:2468] T 3aff9336a59748629accf669219a4490 P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fe3de95e0e3415badd3ac197021f761 in term 0.
I20260807 01:46:20.030435 26743 raft_consensus.cc:3060] T f07aec3c0cac4b28ad53b3ffcb67734b P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:20.030635 26745 raft_consensus.cc:2468] T 215351a405d34d1eab01fa98b8f066d6 P 03e20eec798943849f25342a646c51ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fe3de95e0e3415badd3ac197021f761 in term 1.
I20260807 01:46:20.031878 26577 leader_election.cc:304] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03e20eec798943849f25342a646c51ca, 5fe3de95e0e3415badd3ac197021f761; no voters:
I20260807 01:46:20.032531 26818 raft_consensus.cc:2804] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:20.032699 26577 leader_election.cc:304] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03e20eec798943849f25342a646c51ca, 5fe3de95e0e3415badd3ac197021f761; no voters:
I20260807 01:46:20.032920 26818 raft_consensus.cc:493] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:20.033275 26818 raft_consensus.cc:3060] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:20.033596 26852 raft_consensus.cc:2804] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:20.034147 26852 raft_consensus.cc:697] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Becoming Leader. State: Replica: 5fe3de95e0e3415badd3ac197021f761, State: Running, Role: LEADER
I20260807 01:46:20.035118 26852 consensus_queue.cc:237] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.041210 26818 raft_consensus.cc:515] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.043620 26818 leader_election.cc:290] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [CANDIDATE]: Term 1 election: Requested vote from peers 3c8a6986074c4a15912eeca703de1f91 (127.24.4.1:40495), 03e20eec798943849f25342a646c51ca (127.24.4.3:39659)
I20260807 01:46:20.043798 26743 raft_consensus.cc:2468] T f07aec3c0cac4b28ad53b3ffcb67734b P 03e20eec798943849f25342a646c51ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fe3de95e0e3415badd3ac197021f761 in term 1.
I20260807 01:46:20.044394 26828 raft_consensus.cc:493] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:20.044986 26745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aff9336a59748629accf669219a4490" candidate_uuid: "5fe3de95e0e3415badd3ac197021f761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03e20eec798943849f25342a646c51ca"
I20260807 01:46:20.045624 26745 raft_consensus.cc:3060] T 3aff9336a59748629accf669219a4490 P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:20.045498 26828 raft_consensus.cc:515] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.048772 26828 leader_election.cc:290] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c8a6986074c4a15912eeca703de1f91 (127.24.4.1:40495), 5fe3de95e0e3415badd3ac197021f761 (127.24.4.2:44813)
I20260807 01:46:20.054981 26745 raft_consensus.cc:2468] T 3aff9336a59748629accf669219a4490 P 03e20eec798943849f25342a646c51ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fe3de95e0e3415badd3ac197021f761 in term 1.
I20260807 01:46:20.058579 26515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aff9336a59748629accf669219a4490" candidate_uuid: "5fe3de95e0e3415badd3ac197021f761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c8a6986074c4a15912eeca703de1f91" is_pre_election: true
I20260807 01:46:20.059083 26513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aff9336a59748629accf669219a4490" candidate_uuid: "5fe3de95e0e3415badd3ac197021f761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c8a6986074c4a15912eeca703de1f91"
I20260807 01:46:20.059304 26515 raft_consensus.cc:2468] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fe3de95e0e3415badd3ac197021f761 in term 0.
I20260807 01:46:20.058310 26324 catalog_manager.cc:5697] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fe3de95e0e3415badd3ac197021f761 (127.24.4.2). New cstate: current_term: 1 leader_uuid: "5fe3de95e0e3415badd3ac197021f761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:20.060997 26516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "215351a405d34d1eab01fa98b8f066d6" candidate_uuid: "5fe3de95e0e3415badd3ac197021f761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c8a6986074c4a15912eeca703de1f91"
I20260807 01:46:20.061652 26516 raft_consensus.cc:3060] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:20.061779 26578 leader_election.cc:304] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c8a6986074c4a15912eeca703de1f91, 5fe3de95e0e3415badd3ac197021f761; no voters:
I20260807 01:46:20.063840 26852 raft_consensus.cc:2804] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:20.064380 26852 raft_consensus.cc:697] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Becoming Leader. State: Replica: 5fe3de95e0e3415badd3ac197021f761, State: Running, Role: LEADER
I20260807 01:46:20.065414 26852 consensus_queue.cc:237] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.083684 26818 raft_consensus.cc:493] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:20.084275 26818 raft_consensus.cc:515] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } }
I20260807 01:46:20.086793 26818 leader_election.cc:290] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03e20eec798943849f25342a646c51ca (127.24.4.3:39659), 3c8a6986074c4a15912eeca703de1f91 (127.24.4.1:40495)
I20260807 01:46:20.087704 26818 raft_consensus.cc:493] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:20.088346 26818 raft_consensus.cc:515] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.088619 26515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e9fce5dee804aa7ab8d61ee6528d05a" candidate_uuid: "5fe3de95e0e3415badd3ac197021f761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c8a6986074c4a15912eeca703de1f91" is_pre_election: true
I20260807 01:46:20.089344 26515 raft_consensus.cc:2468] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fe3de95e0e3415badd3ac197021f761 in term 0.
I20260807 01:46:20.090715 26818 leader_election.cc:290] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c8a6986074c4a15912eeca703de1f91 (127.24.4.1:40495), 03e20eec798943849f25342a646c51ca (127.24.4.3:39659)
I20260807 01:46:20.091076 26578 leader_election.cc:304] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c8a6986074c4a15912eeca703de1f91, 5fe3de95e0e3415badd3ac197021f761; no voters:
I20260807 01:46:20.092595 26515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb9ab528b474779946bebdd16dc46ab" candidate_uuid: "5fe3de95e0e3415badd3ac197021f761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c8a6986074c4a15912eeca703de1f91" is_pre_election: true
W20260807 01:46:20.094197 26578 leader_election.cc:343] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c8a6986074c4a15912eeca703de1f91 (127.24.4.1:40495): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:20.094242 26516 raft_consensus.cc:2468] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fe3de95e0e3415badd3ac197021f761 in term 1.
I20260807 01:46:20.094955 26818 raft_consensus.cc:2804] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:20.095336 26818 raft_consensus.cc:493] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:20.095675 26818 raft_consensus.cc:3060] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:20.104348 26577 leader_election.cc:304] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 03e20eec798943849f25342a646c51ca, 5fe3de95e0e3415badd3ac197021f761; no voters: 3c8a6986074c4a15912eeca703de1f91
I20260807 01:46:20.105180 26743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e9fce5dee804aa7ab8d61ee6528d05a" candidate_uuid: "5fe3de95e0e3415badd3ac197021f761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03e20eec798943849f25342a646c51ca" is_pre_election: true
I20260807 01:46:20.105898 26743 raft_consensus.cc:2468] T 0e9fce5dee804aa7ab8d61ee6528d05a P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fe3de95e0e3415badd3ac197021f761 in term 0.
I20260807 01:46:20.106643 26745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb9ab528b474779946bebdd16dc46ab" candidate_uuid: "5fe3de95e0e3415badd3ac197021f761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03e20eec798943849f25342a646c51ca" is_pre_election: true
I20260807 01:46:20.103111 26818 raft_consensus.cc:515] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } }
I20260807 01:46:20.106914 26861 raft_consensus.cc:2804] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:20.107514 26745 raft_consensus.cc:2468] T 1cb9ab528b474779946bebdd16dc46ab P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fe3de95e0e3415badd3ac197021f761 in term 0.
I20260807 01:46:20.112401 26820 tablet_bootstrap.cc:492] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91: No bootstrap required, opened a new log
I20260807 01:46:20.112993 26820 ts_tablet_manager.cc:1403] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91: Time spent bootstrapping tablet: real 0.108s user 0.046s sys 0.052s
I20260807 01:46:20.115676 26577 leader_election.cc:304] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 03e20eec798943849f25342a646c51ca, 5fe3de95e0e3415badd3ac197021f761; no voters: 3c8a6986074c4a15912eeca703de1f91
I20260807 01:46:20.117138 26818 leader_election.cc:290] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [CANDIDATE]: Term 1 election: Requested vote from peers 03e20eec798943849f25342a646c51ca (127.24.4.3:39659), 3c8a6986074c4a15912eeca703de1f91 (127.24.4.1:40495)
I20260807 01:46:20.117456 26516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e9fce5dee804aa7ab8d61ee6528d05a" candidate_uuid: "5fe3de95e0e3415badd3ac197021f761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c8a6986074c4a15912eeca703de1f91"
I20260807 01:46:20.117897 26852 raft_consensus.cc:2804] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:20.122221 26516 raft_consensus.cc:3060] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:20.122103 26852 raft_consensus.cc:493] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:20.123613 26852 raft_consensus.cc:3060] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:20.118566 26745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e9fce5dee804aa7ab8d61ee6528d05a" candidate_uuid: "5fe3de95e0e3415badd3ac197021f761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03e20eec798943849f25342a646c51ca"
I20260807 01:46:20.127899 26745 raft_consensus.cc:3060] T 0e9fce5dee804aa7ab8d61ee6528d05a P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:20.129667 26515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a5fdee17bfc441f9a2ef24cc5b6f2e7" candidate_uuid: "03e20eec798943849f25342a646c51ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c8a6986074c4a15912eeca703de1f91" is_pre_election: true
I20260807 01:46:20.130347 26861 raft_consensus.cc:697] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Becoming Leader. State: Replica: 5fe3de95e0e3415badd3ac197021f761, State: Running, Role: LEADER
I20260807 01:46:20.131054 26515 raft_consensus.cc:2468] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03e20eec798943849f25342a646c51ca in term 0.
I20260807 01:46:20.132047 26861 consensus_queue.cc:237] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.132649 26638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a5fdee17bfc441f9a2ef24cc5b6f2e7" candidate_uuid: "03e20eec798943849f25342a646c51ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fe3de95e0e3415badd3ac197021f761" is_pre_election: true
I20260807 01:46:20.132820 26695 leader_election.cc:304] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03e20eec798943849f25342a646c51ca, 3c8a6986074c4a15912eeca703de1f91; no voters:
I20260807 01:46:20.133352 26638 raft_consensus.cc:2468] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03e20eec798943849f25342a646c51ca in term 0.
I20260807 01:46:20.133734 26828 raft_consensus.cc:2804] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:20.134104 26828 raft_consensus.cc:493] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:20.134475 26828 raft_consensus.cc:3060] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:20.137101 26516 raft_consensus.cc:2468] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fe3de95e0e3415badd3ac197021f761 in term 1.
I20260807 01:46:20.137323 26820 raft_consensus.cc:359] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.138121 26820 raft_consensus.cc:385] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:20.138453 26578 leader_election.cc:304] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c8a6986074c4a15912eeca703de1f91, 5fe3de95e0e3415badd3ac197021f761; no voters:
I20260807 01:46:20.140578 26818 raft_consensus.cc:2804] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:20.141078 26818 raft_consensus.cc:697] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Becoming Leader. State: Replica: 5fe3de95e0e3415badd3ac197021f761, State: Running, Role: LEADER
I20260807 01:46:20.142011 26818 consensus_queue.cc:237] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } }
I20260807 01:46:20.143385 26745 raft_consensus.cc:2468] T 0e9fce5dee804aa7ab8d61ee6528d05a P 03e20eec798943849f25342a646c51ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fe3de95e0e3415badd3ac197021f761 in term 1.
I20260807 01:46:20.143714 26828 raft_consensus.cc:515] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.138466 26820 raft_consensus.cc:740] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c8a6986074c4a15912eeca703de1f91, State: Initialized, Role: FOLLOWER
I20260807 01:46:20.146843 26820 consensus_queue.cc:260] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.145465 26828 leader_election.cc:290] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [CANDIDATE]: Term 1 election: Requested vote from peers 3c8a6986074c4a15912eeca703de1f91 (127.24.4.1:40495), 5fe3de95e0e3415badd3ac197021f761 (127.24.4.2:44813)
I20260807 01:46:20.148618 26638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a5fdee17bfc441f9a2ef24cc5b6f2e7" candidate_uuid: "03e20eec798943849f25342a646c51ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fe3de95e0e3415badd3ac197021f761"
I20260807 01:46:20.148816 26516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a5fdee17bfc441f9a2ef24cc5b6f2e7" candidate_uuid: "03e20eec798943849f25342a646c51ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c8a6986074c4a15912eeca703de1f91"
I20260807 01:46:20.145650 26852 raft_consensus.cc:515] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.149145 26638 raft_consensus.cc:3060] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 5fe3de95e0e3415badd3ac197021f761 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:20.149694 26516 raft_consensus.cc:3060] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:20.150521 26820 ts_tablet_manager.cc:1434] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91: Time spent starting tablet: real 0.037s user 0.004s sys 0.006s
I20260807 01:46:20.160262 26852 leader_election.cc:290] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [CANDIDATE]: Term 1 election: Requested vote from peers 3c8a6986074c4a15912eeca703de1f91 (127.24.4.1:40495), 03e20eec798943849f25342a646c51ca (127.24.4.3:39659)
I20260807 01:46:20.161370 26515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb9ab528b474779946bebdd16dc46ab" candidate_uuid: "5fe3de95e0e3415badd3ac197021f761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c8a6986074c4a15912eeca703de1f91"
I20260807 01:46:20.162127 26515 raft_consensus.cc:3060] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:20.162389 26745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb9ab528b474779946bebdd16dc46ab" candidate_uuid: "5fe3de95e0e3415badd3ac197021f761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03e20eec798943849f25342a646c51ca"
I20260807 01:46:20.163043 26745 raft_consensus.cc:3060] T 1cb9ab528b474779946bebdd16dc46ab P 03e20eec798943849f25342a646c51ca [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:20.168300 26516 raft_consensus.cc:2468] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03e20eec798943849f25342a646c51ca in term 1.
I20260807 01:46:20.169643 26695 leader_election.cc:304] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03e20eec798943849f25342a646c51ca, 3c8a6986074c4a15912eeca703de1f91; no voters:
I20260807 01:46:20.172168 26638 raft_consensus.cc:2468] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 5fe3de95e0e3415badd3ac197021f761 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03e20eec798943849f25342a646c51ca in term 1.
I20260807 01:46:20.174731 26745 raft_consensus.cc:2468] T 1cb9ab528b474779946bebdd16dc46ab P 03e20eec798943849f25342a646c51ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fe3de95e0e3415badd3ac197021f761 in term 1.
I20260807 01:46:20.174849 26515 raft_consensus.cc:2468] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fe3de95e0e3415badd3ac197021f761 in term 1.
I20260807 01:46:20.176277 26828 raft_consensus.cc:2804] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:20.176156 26577 leader_election.cc:304] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03e20eec798943849f25342a646c51ca, 5fe3de95e0e3415badd3ac197021f761; no voters:
I20260807 01:46:20.177698 26861 raft_consensus.cc:2804] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:20.178084 26861 raft_consensus.cc:697] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Becoming Leader. State: Replica: 5fe3de95e0e3415badd3ac197021f761, State: Running, Role: LEADER
I20260807 01:46:20.178787 26861 consensus_queue.cc:237] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.181492 26323 catalog_manager.cc:5697] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fe3de95e0e3415badd3ac197021f761 (127.24.4.2). New cstate: current_term: 1 leader_uuid: "5fe3de95e0e3415badd3ac197021f761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:20.183789 26323 catalog_manager.cc:5697] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fe3de95e0e3415badd3ac197021f761 (127.24.4.2). New cstate: current_term: 1 leader_uuid: "5fe3de95e0e3415badd3ac197021f761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:20.185304 26828 raft_consensus.cc:697] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [term 1 LEADER]: Becoming Leader. State: Replica: 03e20eec798943849f25342a646c51ca, State: Running, Role: LEADER
I20260807 01:46:20.186295 26828 consensus_queue.cc:237] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.186098 26323 catalog_manager.cc:5697] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fe3de95e0e3415badd3ac197021f761 (127.24.4.2). New cstate: current_term: 1 leader_uuid: "5fe3de95e0e3415badd3ac197021f761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:20.195775 26321 catalog_manager.cc:5697] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 03e20eec798943849f25342a646c51ca (127.24.4.3). New cstate: current_term: 1 leader_uuid: "03e20eec798943849f25342a646c51ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:20.203850 26323 catalog_manager.cc:5697] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fe3de95e0e3415badd3ac197021f761 (127.24.4.2). New cstate: current_term: 1 leader_uuid: "5fe3de95e0e3415badd3ac197021f761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:20.341236 26638 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0e9fce5dee804aa7ab8d61ee6528d05a"
dest_uuid: "5fe3de95e0e3415badd3ac197021f761"
mode: GRACEFUL
new_leader_uuid: "3c8a6986074c4a15912eeca703de1f91"
from {username='slave'} at 127.0.0.1:45664
I20260807 01:46:20.341836 26638 raft_consensus.cc:606] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Received request to transfer leadership to TS 3c8a6986074c4a15912eeca703de1f91
I20260807 01:46:20.348438 26638 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1cb9ab528b474779946bebdd16dc46ab"
dest_uuid: "5fe3de95e0e3415badd3ac197021f761"
mode: GRACEFUL
new_leader_uuid: "3c8a6986074c4a15912eeca703de1f91"
from {username='slave'} at 127.0.0.1:45664
I20260807 01:46:20.348963 26638 raft_consensus.cc:606] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Received request to transfer leadership to TS 3c8a6986074c4a15912eeca703de1f91
I20260807 01:46:20.355520 26638 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f07aec3c0cac4b28ad53b3ffcb67734b"
dest_uuid: "5fe3de95e0e3415badd3ac197021f761"
mode: GRACEFUL
new_leader_uuid: "3c8a6986074c4a15912eeca703de1f91"
from {username='slave'} at 127.0.0.1:45664
I20260807 01:46:20.356103 26638 raft_consensus.cc:606] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Received request to transfer leadership to TS 3c8a6986074c4a15912eeca703de1f91
I20260807 01:46:20.361371 26638 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3aff9336a59748629accf669219a4490"
dest_uuid: "5fe3de95e0e3415badd3ac197021f761"
mode: GRACEFUL
new_leader_uuid: "3c8a6986074c4a15912eeca703de1f91"
from {username='slave'} at 127.0.0.1:45664
I20260807 01:46:20.361891 26638 raft_consensus.cc:606] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Received request to transfer leadership to TS 3c8a6986074c4a15912eeca703de1f91
I20260807 01:46:20.367254 26638 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "215351a405d34d1eab01fa98b8f066d6"
dest_uuid: "5fe3de95e0e3415badd3ac197021f761"
mode: GRACEFUL
new_leader_uuid: "3c8a6986074c4a15912eeca703de1f91"
from {username='slave'} at 127.0.0.1:45664
I20260807 01:46:20.367785 26638 raft_consensus.cc:606] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Received request to transfer leadership to TS 3c8a6986074c4a15912eeca703de1f91
I20260807 01:46:20.380913 26745 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9a5fdee17bfc441f9a2ef24cc5b6f2e7"
dest_uuid: "03e20eec798943849f25342a646c51ca"
mode: GRACEFUL
new_leader_uuid: "3c8a6986074c4a15912eeca703de1f91"
from {username='slave'} at 127.0.0.1:34088
I20260807 01:46:20.381438 26745 raft_consensus.cc:606] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [term 1 LEADER]: Received request to transfer leadership to TS 3c8a6986074c4a15912eeca703de1f91
I20260807 01:46:20.430104 26861 consensus_queue.cc:1048] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:20.450177 26861 consensus_queue.cc:1048] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:20.463771 26818 raft_consensus.cc:993] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761: : Instructing follower 3c8a6986074c4a15912eeca703de1f91 to start an election
I20260807 01:46:20.464313 26852 raft_consensus.cc:1081] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Signalling peer 3c8a6986074c4a15912eeca703de1f91 to start an election
I20260807 01:46:20.466324 26515 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "215351a405d34d1eab01fa98b8f066d6"
dest_uuid: "3c8a6986074c4a15912eeca703de1f91"
from {username='slave'} at 127.0.0.1:49228
I20260807 01:46:20.467442 26515 raft_consensus.cc:493] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:46:20.467819 26515 raft_consensus.cc:3060] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:20.473994 26515 raft_consensus.cc:515] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.476605 26515 leader_election.cc:290] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91 [CANDIDATE]: Term 2 election: Requested vote from peers 5fe3de95e0e3415badd3ac197021f761 (127.24.4.2:44813), 03e20eec798943849f25342a646c51ca (127.24.4.3:39659)
I20260807 01:46:20.494820 26745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "215351a405d34d1eab01fa98b8f066d6" candidate_uuid: "3c8a6986074c4a15912eeca703de1f91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "03e20eec798943849f25342a646c51ca"
I20260807 01:46:20.495507 26745 raft_consensus.cc:3060] T 215351a405d34d1eab01fa98b8f066d6 P 03e20eec798943849f25342a646c51ca [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:20.496814 26638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "215351a405d34d1eab01fa98b8f066d6" candidate_uuid: "3c8a6986074c4a15912eeca703de1f91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5fe3de95e0e3415badd3ac197021f761"
I20260807 01:46:20.497771 26638 raft_consensus.cc:3055] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:20.498315 26638 raft_consensus.cc:740] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5fe3de95e0e3415badd3ac197021f761, State: Running, Role: LEADER
I20260807 01:46:20.499117 26638 consensus_queue.cc:260] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.500355 26638 raft_consensus.cc:3060] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:20.502669 26745 raft_consensus.cc:2468] T 215351a405d34d1eab01fa98b8f066d6 P 03e20eec798943849f25342a646c51ca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c8a6986074c4a15912eeca703de1f91 in term 2.
I20260807 01:46:20.503901 26450 leader_election.cc:304] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03e20eec798943849f25342a646c51ca, 3c8a6986074c4a15912eeca703de1f91; no voters:
I20260807 01:46:20.504776 26829 raft_consensus.cc:2804] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:46:20.505914 26829 raft_consensus.cc:697] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91 [term 2 LEADER]: Becoming Leader. State: Replica: 3c8a6986074c4a15912eeca703de1f91, State: Running, Role: LEADER
I20260807 01:46:20.506740 26638 raft_consensus.cc:2468] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c8a6986074c4a15912eeca703de1f91 in term 2.
I20260807 01:46:20.506834 26829 consensus_queue.cc:237] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.513733 26322 catalog_manager.cc:5697] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91 reported cstate change: term changed from 1 to 2, leader changed from 5fe3de95e0e3415badd3ac197021f761 (127.24.4.2) to 3c8a6986074c4a15912eeca703de1f91 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "3c8a6986074c4a15912eeca703de1f91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:20.559900 26852 consensus_queue.cc:1048] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:20.567888 26818 raft_consensus.cc:993] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761: : Instructing follower 3c8a6986074c4a15912eeca703de1f91 to start an election
I20260807 01:46:20.568434 26818 raft_consensus.cc:1081] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Signalling peer 3c8a6986074c4a15912eeca703de1f91 to start an election
I20260807 01:46:20.575235 26861 consensus_queue.cc:1048] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:20.577876 26870 consensus_queue.cc:1048] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:20.578305 26515 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f07aec3c0cac4b28ad53b3ffcb67734b"
dest_uuid: "3c8a6986074c4a15912eeca703de1f91"
from {username='slave'} at 127.0.0.1:49228
I20260807 01:46:20.578908 26515 raft_consensus.cc:493] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:46:20.579313 26515 raft_consensus.cc:3060] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:20.580149 26870 consensus_queue.cc:1048] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:20.588006 26515 raft_consensus.cc:515] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.591128 26638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f07aec3c0cac4b28ad53b3ffcb67734b" candidate_uuid: "3c8a6986074c4a15912eeca703de1f91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5fe3de95e0e3415badd3ac197021f761"
I20260807 01:46:20.591768 26638 raft_consensus.cc:3055] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:20.592082 26638 raft_consensus.cc:740] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5fe3de95e0e3415badd3ac197021f761, State: Running, Role: LEADER
I20260807 01:46:20.592895 26638 consensus_queue.cc:260] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.593873 26638 raft_consensus.cc:3060] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:20.607702 26513 raft_consensus.cc:3060] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:20.608139 26745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f07aec3c0cac4b28ad53b3ffcb67734b" candidate_uuid: "3c8a6986074c4a15912eeca703de1f91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "03e20eec798943849f25342a646c51ca"
I20260807 01:46:20.608762 26745 raft_consensus.cc:3060] T f07aec3c0cac4b28ad53b3ffcb67734b P 03e20eec798943849f25342a646c51ca [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:20.610991 26515 leader_election.cc:290] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91 [CANDIDATE]: Term 2 election: Requested vote from peers 5fe3de95e0e3415badd3ac197021f761 (127.24.4.2:44813), 03e20eec798943849f25342a646c51ca (127.24.4.3:39659)
I20260807 01:46:20.602717 26638 raft_consensus.cc:2468] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c8a6986074c4a15912eeca703de1f91 in term 2.
I20260807 01:46:20.612876 26451 leader_election.cc:304] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c8a6986074c4a15912eeca703de1f91, 5fe3de95e0e3415badd3ac197021f761; no voters:
I20260807 01:46:20.613976 26829 raft_consensus.cc:2804] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:46:20.615161 26818 consensus_queue.cc:1048] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:20.618198 26745 raft_consensus.cc:2468] T f07aec3c0cac4b28ad53b3ffcb67734b P 03e20eec798943849f25342a646c51ca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c8a6986074c4a15912eeca703de1f91 in term 2.
I20260807 01:46:20.623818 26818 consensus_queue.cc:1048] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:20.614500 26829 raft_consensus.cc:697] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91 [term 2 LEADER]: Becoming Leader. State: Replica: 3c8a6986074c4a15912eeca703de1f91, State: Running, Role: LEADER
I20260807 01:46:20.626813 26829 consensus_queue.cc:237] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.632212 26861 raft_consensus.cc:993] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761: : Instructing follower 3c8a6986074c4a15912eeca703de1f91 to start an election
I20260807 01:46:20.632748 26861 raft_consensus.cc:1081] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Signalling peer 3c8a6986074c4a15912eeca703de1f91 to start an election
I20260807 01:46:20.637072 26516 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0e9fce5dee804aa7ab8d61ee6528d05a"
dest_uuid: "3c8a6986074c4a15912eeca703de1f91"
from {username='slave'} at 127.0.0.1:49228
I20260807 01:46:20.637665 26516 raft_consensus.cc:493] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:46:20.638043 26516 raft_consensus.cc:3060] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:20.645218 26516 raft_consensus.cc:515] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } }
I20260807 01:46:20.648562 26743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e9fce5dee804aa7ab8d61ee6528d05a" candidate_uuid: "3c8a6986074c4a15912eeca703de1f91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "03e20eec798943849f25342a646c51ca"
I20260807 01:46:20.649197 26743 raft_consensus.cc:3060] T 0e9fce5dee804aa7ab8d61ee6528d05a P 03e20eec798943849f25342a646c51ca [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:20.650324 26516 leader_election.cc:290] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91 [CANDIDATE]: Term 2 election: Requested vote from peers 03e20eec798943849f25342a646c51ca (127.24.4.3:39659), 5fe3de95e0e3415badd3ac197021f761 (127.24.4.2:44813)
I20260807 01:46:20.651005 26638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e9fce5dee804aa7ab8d61ee6528d05a" candidate_uuid: "3c8a6986074c4a15912eeca703de1f91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5fe3de95e0e3415badd3ac197021f761"
I20260807 01:46:20.651621 26638 raft_consensus.cc:3055] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:20.651975 26638 raft_consensus.cc:740] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5fe3de95e0e3415badd3ac197021f761, State: Running, Role: LEADER
I20260807 01:46:20.652781 26638 consensus_queue.cc:260] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } }
I20260807 01:46:20.654053 26638 raft_consensus.cc:3060] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:20.655462 26852 raft_consensus.cc:993] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761: : Instructing follower 3c8a6986074c4a15912eeca703de1f91 to start an election
I20260807 01:46:20.655965 26870 raft_consensus.cc:1081] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Signalling peer 3c8a6986074c4a15912eeca703de1f91 to start an election
I20260807 01:46:20.656574 26743 raft_consensus.cc:2468] T 0e9fce5dee804aa7ab8d61ee6528d05a P 03e20eec798943849f25342a646c51ca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c8a6986074c4a15912eeca703de1f91 in term 2.
I20260807 01:46:20.661224 26450 leader_election.cc:304] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03e20eec798943849f25342a646c51ca, 3c8a6986074c4a15912eeca703de1f91; no voters:
I20260807 01:46:20.661301 26638 raft_consensus.cc:2468] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c8a6986074c4a15912eeca703de1f91 in term 2.
I20260807 01:46:20.657894 26515 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3aff9336a59748629accf669219a4490"
dest_uuid: "3c8a6986074c4a15912eeca703de1f91"
from {username='slave'} at 127.0.0.1:49228
I20260807 01:46:20.661979 26829 raft_consensus.cc:2804] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:46:20.662199 26515 raft_consensus.cc:493] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:46:20.662704 26829 raft_consensus.cc:697] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91 [term 2 LEADER]: Becoming Leader. State: Replica: 3c8a6986074c4a15912eeca703de1f91, State: Running, Role: LEADER
I20260807 01:46:20.662628 26515 raft_consensus.cc:3060] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:20.663595 26829 consensus_queue.cc:237] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } }
I20260807 01:46:20.667482 26323 catalog_manager.cc:5697] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91 reported cstate change: term changed from 1 to 2, leader changed from 5fe3de95e0e3415badd3ac197021f761 (127.24.4.2) to 3c8a6986074c4a15912eeca703de1f91 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "3c8a6986074c4a15912eeca703de1f91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:20.668871 26852 consensus_queue.cc:1048] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:20.675235 26515 raft_consensus.cc:515] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.678964 26638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aff9336a59748629accf669219a4490" candidate_uuid: "3c8a6986074c4a15912eeca703de1f91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5fe3de95e0e3415badd3ac197021f761"
I20260807 01:46:20.679622 26638 raft_consensus.cc:3055] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:20.679978 26638 raft_consensus.cc:740] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5fe3de95e0e3415badd3ac197021f761, State: Running, Role: LEADER
I20260807 01:46:20.680843 26638 consensus_queue.cc:260] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.683627 26515 leader_election.cc:290] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91 [CANDIDATE]: Term 2 election: Requested vote from peers 5fe3de95e0e3415badd3ac197021f761 (127.24.4.2:44813), 03e20eec798943849f25342a646c51ca (127.24.4.3:39659)
I20260807 01:46:20.686383 26743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aff9336a59748629accf669219a4490" candidate_uuid: "3c8a6986074c4a15912eeca703de1f91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "03e20eec798943849f25342a646c51ca"
I20260807 01:46:20.687021 26743 raft_consensus.cc:3060] T 3aff9336a59748629accf669219a4490 P 03e20eec798943849f25342a646c51ca [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:20.687934 26324 catalog_manager.cc:5697] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91 reported cstate change: term changed from 1 to 2, leader changed from 5fe3de95e0e3415badd3ac197021f761 (127.24.4.2) to 3c8a6986074c4a15912eeca703de1f91 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "3c8a6986074c4a15912eeca703de1f91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:20.687825 26638 raft_consensus.cc:3060] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:20.702138 26861 consensus_queue.cc:1048] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:20.703912 26743 raft_consensus.cc:2468] T 3aff9336a59748629accf669219a4490 P 03e20eec798943849f25342a646c51ca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c8a6986074c4a15912eeca703de1f91 in term 2.
I20260807 01:46:20.704416 26638 raft_consensus.cc:2468] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c8a6986074c4a15912eeca703de1f91 in term 2.
I20260807 01:46:20.705109 26450 leader_election.cc:304] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03e20eec798943849f25342a646c51ca, 3c8a6986074c4a15912eeca703de1f91; no voters:
I20260807 01:46:20.705868 26829 raft_consensus.cc:2804] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:46:20.707480 26829 raft_consensus.cc:697] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91 [term 2 LEADER]: Becoming Leader. State: Replica: 3c8a6986074c4a15912eeca703de1f91, State: Running, Role: LEADER
I20260807 01:46:20.708590 26829 consensus_queue.cc:237] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.710781 26852 raft_consensus.cc:993] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761: : Instructing follower 3c8a6986074c4a15912eeca703de1f91 to start an election
I20260807 01:46:20.711242 26861 raft_consensus.cc:1081] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Signalling peer 3c8a6986074c4a15912eeca703de1f91 to start an election
I20260807 01:46:20.712926 26515 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1cb9ab528b474779946bebdd16dc46ab"
dest_uuid: "3c8a6986074c4a15912eeca703de1f91"
from {username='slave'} at 127.0.0.1:49228
I20260807 01:46:20.713426 26515 raft_consensus.cc:493] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:46:20.713721 26515 raft_consensus.cc:3060] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:20.719717 26515 raft_consensus.cc:515] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.721546 26515 leader_election.cc:290] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91 [CANDIDATE]: Term 2 election: Requested vote from peers 5fe3de95e0e3415badd3ac197021f761 (127.24.4.2:44813), 03e20eec798943849f25342a646c51ca (127.24.4.3:39659)
I20260807 01:46:20.722460 26638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb9ab528b474779946bebdd16dc46ab" candidate_uuid: "3c8a6986074c4a15912eeca703de1f91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5fe3de95e0e3415badd3ac197021f761"
I20260807 01:46:20.722795 26743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb9ab528b474779946bebdd16dc46ab" candidate_uuid: "3c8a6986074c4a15912eeca703de1f91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "03e20eec798943849f25342a646c51ca"
I20260807 01:46:20.722991 26638 raft_consensus.cc:3055] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:20.723366 26638 raft_consensus.cc:740] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5fe3de95e0e3415badd3ac197021f761, State: Running, Role: LEADER
I20260807 01:46:20.723379 26743 raft_consensus.cc:3060] T 1cb9ab528b474779946bebdd16dc46ab P 03e20eec798943849f25342a646c51ca [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:20.724303 26638 consensus_queue.cc:260] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.725291 26638 raft_consensus.cc:3060] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:20.726035 26323 catalog_manager.cc:5697] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91 reported cstate change: term changed from 1 to 2, leader changed from 5fe3de95e0e3415badd3ac197021f761 (127.24.4.2) to 3c8a6986074c4a15912eeca703de1f91 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "3c8a6986074c4a15912eeca703de1f91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:20.730111 26828 consensus_queue.cc:1048] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:20.732332 26743 raft_consensus.cc:2468] T 1cb9ab528b474779946bebdd16dc46ab P 03e20eec798943849f25342a646c51ca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c8a6986074c4a15912eeca703de1f91 in term 2.
I20260807 01:46:20.733027 26638 raft_consensus.cc:2468] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c8a6986074c4a15912eeca703de1f91 in term 2.
I20260807 01:46:20.733939 26450 leader_election.cc:304] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03e20eec798943849f25342a646c51ca, 3c8a6986074c4a15912eeca703de1f91; no voters:
I20260807 01:46:20.736322 26829 raft_consensus.cc:2804] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:46:20.736951 26829 raft_consensus.cc:697] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91 [term 2 LEADER]: Becoming Leader. State: Replica: 3c8a6986074c4a15912eeca703de1f91, State: Running, Role: LEADER
I20260807 01:46:20.738109 26829 consensus_queue.cc:237] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.747435 26913 consensus_queue.cc:1048] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:20.752825 26324 catalog_manager.cc:5697] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91 reported cstate change: term changed from 1 to 2, leader changed from 5fe3de95e0e3415badd3ac197021f761 (127.24.4.2) to 3c8a6986074c4a15912eeca703de1f91 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "3c8a6986074c4a15912eeca703de1f91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:20.764989 26914 raft_consensus.cc:993] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca: : Instructing follower 3c8a6986074c4a15912eeca703de1f91 to start an election
I20260807 01:46:20.765472 26914 raft_consensus.cc:1081] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [term 1 LEADER]: Signalling peer 3c8a6986074c4a15912eeca703de1f91 to start an election
I20260807 01:46:20.767179 26515 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9a5fdee17bfc441f9a2ef24cc5b6f2e7"
dest_uuid: "3c8a6986074c4a15912eeca703de1f91"
from {username='slave'} at 127.0.0.1:49234
I20260807 01:46:20.767664 26515 raft_consensus.cc:493] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:46:20.767964 26515 raft_consensus.cc:3060] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:20.773994 26515 raft_consensus.cc:515] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.776767 26515 leader_election.cc:290] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91 [CANDIDATE]: Term 2 election: Requested vote from peers 5fe3de95e0e3415badd3ac197021f761 (127.24.4.2:44813), 03e20eec798943849f25342a646c51ca (127.24.4.3:39659)
I20260807 01:46:20.777828 26637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a5fdee17bfc441f9a2ef24cc5b6f2e7" candidate_uuid: "3c8a6986074c4a15912eeca703de1f91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5fe3de95e0e3415badd3ac197021f761"
I20260807 01:46:20.778061 26743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a5fdee17bfc441f9a2ef24cc5b6f2e7" candidate_uuid: "3c8a6986074c4a15912eeca703de1f91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "03e20eec798943849f25342a646c51ca"
I20260807 01:46:20.778499 26637 raft_consensus.cc:3060] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 5fe3de95e0e3415badd3ac197021f761 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:20.778728 26743 raft_consensus.cc:3055] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:20.779071 26743 raft_consensus.cc:740] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 03e20eec798943849f25342a646c51ca, State: Running, Role: LEADER
I20260807 01:46:20.779806 26743 consensus_queue.cc:260] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.781019 26743 raft_consensus.cc:3060] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:20.785674 26637 raft_consensus.cc:2468] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 5fe3de95e0e3415badd3ac197021f761 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c8a6986074c4a15912eeca703de1f91 in term 2.
I20260807 01:46:20.787420 26451 leader_election.cc:304] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c8a6986074c4a15912eeca703de1f91, 5fe3de95e0e3415badd3ac197021f761; no voters:
I20260807 01:46:20.788275 26829 raft_consensus.cc:2804] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:46:20.787369 26743 raft_consensus.cc:2468] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c8a6986074c4a15912eeca703de1f91 in term 2.
I20260807 01:46:20.788776 26829 raft_consensus.cc:697] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91 [term 2 LEADER]: Becoming Leader. State: Replica: 3c8a6986074c4a15912eeca703de1f91, State: Running, Role: LEADER
I20260807 01:46:20.789613 26829 consensus_queue.cc:237] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } }
I20260807 01:46:20.796245 26323 catalog_manager.cc:5697] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91 reported cstate change: term changed from 1 to 2, leader changed from 03e20eec798943849f25342a646c51ca (127.24.4.3) to 3c8a6986074c4a15912eeca703de1f91 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "3c8a6986074c4a15912eeca703de1f91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c8a6986074c4a15912eeca703de1f91" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 40495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fe3de95e0e3415badd3ac197021f761" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 44813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03e20eec798943849f25342a646c51ca" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 39659 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:20.797410 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
W20260807 01:46:20.803462 24592 auto_leader_rebalancer.cc:456] leader transfer for tablet 0e9fce5dee804aa7ab8d61ee6528d05a from 3c8a6986074c4a15912eeca703de1f91 to 03e20eec798943849f25342a646c51ca failed: Service unavailable: TEST: forced leader step-down failure
W20260807 01:46:20.804049 24592 auto_leader_rebalancer.cc:456] leader transfer for tablet 215351a405d34d1eab01fa98b8f066d6 from 3c8a6986074c4a15912eeca703de1f91 to 5fe3de95e0e3415badd3ac197021f761 failed: Service unavailable: TEST: forced leader step-down failure
W20260807 01:46:20.804610 24592 auto_leader_rebalancer.cc:456] leader transfer for tablet 3aff9336a59748629accf669219a4490 from 3c8a6986074c4a15912eeca703de1f91 to 5fe3de95e0e3415badd3ac197021f761 failed: Service unavailable: TEST: forced leader step-down failure
I20260807 01:46:20.804858 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:46:20.805204 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:46:20.805624 24592 tablet_server.cc:179] TabletServer@127.24.4.1:0 shutting down...
I20260807 01:46:20.828729 24592 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260807 01:46:20.829537 24592 tablet_replica.cc:333] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91: stopping tablet replica
I20260807 01:46:20.830147 24592 raft_consensus.cc:2243] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91 [term 2 LEADER]: Raft consensus shutting down.
I20260807 01:46:20.830828 24592 pending_rounds.cc:70] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91: Trying to abort 1 pending ops.
I20260807 01:46:20.831092 24592 pending_rounds.cc:77] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7315731172320669696 op_type: NO_OP noop_request { }
I20260807 01:46:20.831432 24592 raft_consensus.cc:2889] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260807 01:46:20.831700 24592 raft_consensus.cc:2272] T 1cb9ab528b474779946bebdd16dc46ab P 3c8a6986074c4a15912eeca703de1f91 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:20.834363 24592 tablet_replica.cc:333] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91: stopping tablet replica
I20260807 01:46:20.835121 24592 raft_consensus.cc:2243] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91 [term 2 LEADER]: Raft consensus shutting down.
I20260807 01:46:20.835716 24592 pending_rounds.cc:70] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91: Trying to abort 1 pending ops.
I20260807 01:46:20.835887 24592 pending_rounds.cc:77] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7315731171366363136 op_type: NO_OP noop_request { }
I20260807 01:46:20.836163 24592 raft_consensus.cc:2889] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260807 01:46:20.836397 24592 raft_consensus.cc:2272] T 215351a405d34d1eab01fa98b8f066d6 P 3c8a6986074c4a15912eeca703de1f91 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:20.838518 24592 tablet_replica.cc:333] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91: stopping tablet replica
I20260807 01:46:20.839061 24592 raft_consensus.cc:2243] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91 [term 2 LEADER]: Raft consensus shutting down.
I20260807 01:46:20.839602 24592 pending_rounds.cc:70] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91: Trying to abort 1 pending ops.
I20260807 01:46:20.839754 24592 pending_rounds.cc:77] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7315731171856707584 op_type: NO_OP noop_request { }
I20260807 01:46:20.839972 24592 raft_consensus.cc:2889] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260807 01:46:20.840183 24592 raft_consensus.cc:2272] T f07aec3c0cac4b28ad53b3ffcb67734b P 3c8a6986074c4a15912eeca703de1f91 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:20.842422 24592 tablet_replica.cc:333] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91: stopping tablet replica
I20260807 01:46:20.842969 24592 raft_consensus.cc:2243] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91 [term 2 LEADER]: Raft consensus shutting down.
I20260807 01:46:20.843647 24592 pending_rounds.cc:70] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91: Trying to abort 1 pending ops.
I20260807 01:46:20.843822 24592 pending_rounds.cc:77] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7315731172009160704 op_type: NO_OP noop_request { }
I20260807 01:46:20.844077 24592 raft_consensus.cc:2889] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260807 01:46:20.844334 24592 raft_consensus.cc:2272] T 0e9fce5dee804aa7ab8d61ee6528d05a P 3c8a6986074c4a15912eeca703de1f91 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:20.846486 24592 tablet_replica.cc:333] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91: stopping tablet replica
I20260807 01:46:20.846976 24592 raft_consensus.cc:2243] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91 [term 2 LEADER]: Raft consensus shutting down.
I20260807 01:46:20.847532 24592 pending_rounds.cc:70] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91: Trying to abort 1 pending ops.
I20260807 01:46:20.847714 24592 pending_rounds.cc:77] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7315731172523073536 op_type: NO_OP noop_request { }
I20260807 01:46:20.847968 24592 raft_consensus.cc:2889] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260807 01:46:20.848223 24592 raft_consensus.cc:2272] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 3c8a6986074c4a15912eeca703de1f91 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:20.850455 24592 tablet_replica.cc:333] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91: stopping tablet replica
I20260807 01:46:20.850945 24592 raft_consensus.cc:2243] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91 [term 2 LEADER]: Raft consensus shutting down.
I20260807 01:46:20.851503 24592 pending_rounds.cc:70] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91: Trying to abort 1 pending ops.
I20260807 01:46:20.851675 24592 pending_rounds.cc:77] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7315731172194336768 op_type: NO_OP noop_request { }
I20260807 01:46:20.851934 24592 raft_consensus.cc:2889] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260807 01:46:20.852195 24592 raft_consensus.cc:2272] T 3aff9336a59748629accf669219a4490 P 3c8a6986074c4a15912eeca703de1f91 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:20.877769 24592 tablet_server.cc:196] TabletServer@127.24.4.1:0 shutdown complete.
I20260807 01:46:20.891420 24592 tablet_server.cc:179] TabletServer@127.24.4.2:0 shutting down...
I20260807 01:46:20.912822 24592 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260807 01:46:20.913463 24592 tablet_replica.cc:333] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761: stopping tablet replica
I20260807 01:46:20.914039 24592 raft_consensus.cc:2243] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:20.914510 24592 raft_consensus.cc:2272] T 1cb9ab528b474779946bebdd16dc46ab P 5fe3de95e0e3415badd3ac197021f761 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:20.916484 24592 tablet_replica.cc:333] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761: stopping tablet replica
I20260807 01:46:20.916999 24592 raft_consensus.cc:2243] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:20.917352 24592 raft_consensus.cc:2272] T 0e9fce5dee804aa7ab8d61ee6528d05a P 5fe3de95e0e3415badd3ac197021f761 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:20.919265 24592 tablet_replica.cc:333] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761: stopping tablet replica
I20260807 01:46:20.919781 24592 raft_consensus.cc:2243] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:20.920176 24592 raft_consensus.cc:2272] T 215351a405d34d1eab01fa98b8f066d6 P 5fe3de95e0e3415badd3ac197021f761 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:20.922060 24592 tablet_replica.cc:333] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761: stopping tablet replica
I20260807 01:46:20.922582 24592 raft_consensus.cc:2243] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:20.922974 24592 raft_consensus.cc:2272] T f07aec3c0cac4b28ad53b3ffcb67734b P 5fe3de95e0e3415badd3ac197021f761 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:20.924981 24592 tablet_replica.cc:333] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761: stopping tablet replica
I20260807 01:46:20.925467 24592 raft_consensus.cc:2243] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:20.925879 24592 raft_consensus.cc:2272] T 3aff9336a59748629accf669219a4490 P 5fe3de95e0e3415badd3ac197021f761 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:20.927871 24592 tablet_replica.cc:333] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 5fe3de95e0e3415badd3ac197021f761: stopping tablet replica
I20260807 01:46:20.928445 24592 raft_consensus.cc:2243] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 5fe3de95e0e3415badd3ac197021f761 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:20.928969 24592 raft_consensus.cc:2272] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 5fe3de95e0e3415badd3ac197021f761 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:20.944314 24592 tablet_server.cc:196] TabletServer@127.24.4.2:0 shutdown complete.
I20260807 01:46:20.956637 24592 tablet_server.cc:179] TabletServer@127.24.4.3:0 shutting down...
I20260807 01:46:20.977077 24592 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260807 01:46:20.977694 24592 tablet_replica.cc:333] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca: stopping tablet replica
I20260807 01:46:20.978317 24592 raft_consensus.cc:2243] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:20.978793 24592 raft_consensus.cc:2272] T 9a5fdee17bfc441f9a2ef24cc5b6f2e7 P 03e20eec798943849f25342a646c51ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:20.980857 24592 tablet_replica.cc:333] T f07aec3c0cac4b28ad53b3ffcb67734b P 03e20eec798943849f25342a646c51ca: stopping tablet replica
I20260807 01:46:20.981393 24592 raft_consensus.cc:2243] T f07aec3c0cac4b28ad53b3ffcb67734b P 03e20eec798943849f25342a646c51ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:20.981797 24592 raft_consensus.cc:2272] T f07aec3c0cac4b28ad53b3ffcb67734b P 03e20eec798943849f25342a646c51ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:20.984059 24592 tablet_replica.cc:333] T 215351a405d34d1eab01fa98b8f066d6 P 03e20eec798943849f25342a646c51ca: stopping tablet replica
I20260807 01:46:20.984583 24592 raft_consensus.cc:2243] T 215351a405d34d1eab01fa98b8f066d6 P 03e20eec798943849f25342a646c51ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:20.984987 24592 raft_consensus.cc:2272] T 215351a405d34d1eab01fa98b8f066d6 P 03e20eec798943849f25342a646c51ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:20.987116 24592 tablet_replica.cc:333] T 0e9fce5dee804aa7ab8d61ee6528d05a P 03e20eec798943849f25342a646c51ca: stopping tablet replica
I20260807 01:46:20.987617 24592 raft_consensus.cc:2243] T 0e9fce5dee804aa7ab8d61ee6528d05a P 03e20eec798943849f25342a646c51ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:20.988019 24592 raft_consensus.cc:2272] T 0e9fce5dee804aa7ab8d61ee6528d05a P 03e20eec798943849f25342a646c51ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:20.990145 24592 tablet_replica.cc:333] T 3aff9336a59748629accf669219a4490 P 03e20eec798943849f25342a646c51ca: stopping tablet replica
I20260807 01:46:20.990680 24592 raft_consensus.cc:2243] T 3aff9336a59748629accf669219a4490 P 03e20eec798943849f25342a646c51ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:20.991075 24592 raft_consensus.cc:2272] T 3aff9336a59748629accf669219a4490 P 03e20eec798943849f25342a646c51ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:20.993160 24592 tablet_replica.cc:333] T 1cb9ab528b474779946bebdd16dc46ab P 03e20eec798943849f25342a646c51ca: stopping tablet replica
I20260807 01:46:20.993623 24592 raft_consensus.cc:2243] T 1cb9ab528b474779946bebdd16dc46ab P 03e20eec798943849f25342a646c51ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:20.994102 24592 raft_consensus.cc:2272] T 1cb9ab528b474779946bebdd16dc46ab P 03e20eec798943849f25342a646c51ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:21.018579 24592 tablet_server.cc:196] TabletServer@127.24.4.3:0 shutdown complete.
I20260807 01:46:21.032264 24592 master.cc:562] Master@127.24.4.62:46391 shutting down...
I20260807 01:46:21.047766 24592 raft_consensus.cc:2243] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814 [term 1 LEADER]: Raft consensus shutting down.
I20260807 01:46:21.048405 24592 raft_consensus.cc:2272] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:21.048755 24592 tablet_replica.cc:333] T 00000000000000000000000000000000 P a6c711799ea8410192475e9f80398814: stopping tablet replica
I20260807 01:46:21.067600 24592 master.cc:584] Master@127.24.4.62:46391 shutdown complete.
[ OK ] LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass (2249 ms)
[ RUN ] LeaderRebalancerTest.AddTserver
I20260807 01:46:21.096817 24592 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.24.4.62:35633
I20260807 01:46:21.097918 24592 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 01:46:21.103229 26921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 01:46:21.104848 26924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 01:46:21.104867 26922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:46:21.105911 24592 server_base.cc:1061] running on GCE node
I20260807 01:46:21.106724 24592 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 01:46:21.106920 24592 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 01:46:21.107064 24592 hybrid_clock.cc:648] HybridClock initialized: now 1786067181107047 us; error 0 us; skew 500 ppm
I20260807 01:46:21.107573 24592 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 01:46:21.110107 24592 webserver.cc:533] Webserver started at http://127.24.4.62:35647/ using document root <none> and password file <none>
I20260807 01:46:21.110649 24592 fs_manager.cc:362] Metadata directory not provided
I20260807 01:46:21.110847 24592 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 01:46:21.111102 24592 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 01:46:21.112479 24592 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/master-0-root/instance:
uuid: "31e8851a2fee40aa9ea5af43f94890df"
format_stamp: "Formatted at 2026-08-07 01:46:21 on dist-test-slave-zj8q"
I20260807 01:46:21.118717 24592 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260807 01:46:21.122673 26930 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 01:46:21.123522 24592 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 01:46:21.123813 24592 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/master-0-root
uuid: "31e8851a2fee40aa9ea5af43f94890df"
format_stamp: "Formatted at 2026-08-07 01:46:21 on dist-test-slave-zj8q"
I20260807 01:46:21.124102 24592 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 01:46:21.144865 24592 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 01:46:21.145987 24592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 01:46:21.189778 24592 rpc_server.cc:307] RPC server started. Bound to: 127.24.4.62:35633
I20260807 01:46:21.190073 27011 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.4.62:35633 every 8 connection(s)
I20260807 01:46:21.195734 27013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:21.209090 27013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df: Bootstrap starting.
I20260807 01:46:21.213752 27013 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:21.218235 27013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df: No bootstrap required, opened a new log
I20260807 01:46:21.220422 27013 raft_consensus.cc:359] T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e8851a2fee40aa9ea5af43f94890df" member_type: VOTER }
I20260807 01:46:21.220853 27013 raft_consensus.cc:385] T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:21.221095 27013 raft_consensus.cc:740] T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31e8851a2fee40aa9ea5af43f94890df, State: Initialized, Role: FOLLOWER
I20260807 01:46:21.221693 27013 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e8851a2fee40aa9ea5af43f94890df" member_type: VOTER }
I20260807 01:46:21.222237 27013 raft_consensus.cc:399] T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 01:46:21.222483 27013 raft_consensus.cc:493] T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 01:46:21.223022 27013 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:21.228650 27013 raft_consensus.cc:515] T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e8851a2fee40aa9ea5af43f94890df" member_type: VOTER }
I20260807 01:46:21.229288 27013 leader_election.cc:304] T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31e8851a2fee40aa9ea5af43f94890df; no voters:
I20260807 01:46:21.230532 27013 leader_election.cc:290] T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 01:46:21.230882 27019 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:21.232220 27019 raft_consensus.cc:697] T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df [term 1 LEADER]: Becoming Leader. State: Replica: 31e8851a2fee40aa9ea5af43f94890df, State: Running, Role: LEADER
I20260807 01:46:21.232923 27019 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e8851a2fee40aa9ea5af43f94890df" member_type: VOTER }
I20260807 01:46:21.233551 27013 sys_catalog.cc:565] T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df [sys.catalog]: configured and running, proceeding with master startup.
I20260807 01:46:21.236014 27021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31e8851a2fee40aa9ea5af43f94890df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e8851a2fee40aa9ea5af43f94890df" member_type: VOTER } }
I20260807 01:46:21.236054 27022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31e8851a2fee40aa9ea5af43f94890df. Latest consensus state: current_term: 1 leader_uuid: "31e8851a2fee40aa9ea5af43f94890df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e8851a2fee40aa9ea5af43f94890df" member_type: VOTER } }
I20260807 01:46:21.236707 27021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df [sys.catalog]: This master's current role is: LEADER
I20260807 01:46:21.237030 27022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df [sys.catalog]: This master's current role is: LEADER
I20260807 01:46:21.243053 27031 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260807 01:46:21.250767 27031 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260807 01:46:21.253451 24592 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 01:46:21.260772 27031 catalog_manager.cc:1383] Generated new cluster ID: 3ef08cdb5ebf46f0b60129286cb15479
I20260807 01:46:21.261127 27031 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260807 01:46:21.274505 27031 catalog_manager.cc:1406] Generated new certificate authority record
I20260807 01:46:21.275911 27031 catalog_manager.cc:1540] Loading token signing keys...
I20260807 01:46:21.290007 27031 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df: Generated new TSK 0
I20260807 01:46:21.290696 27031 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260807 01:46:21.321452 24592 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 01:46:21.327770 27047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 01:46:21.328676 27048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 01:46:21.330384 27052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:46:21.331086 24592 server_base.cc:1061] running on GCE node
I20260807 01:46:21.331930 24592 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 01:46:21.332126 24592 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 01:46:21.332289 24592 hybrid_clock.cc:648] HybridClock initialized: now 1786067181332273 us; error 0 us; skew 500 ppm
I20260807 01:46:21.332844 24592 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 01:46:21.335484 24592 webserver.cc:533] Webserver started at http://127.24.4.1:42529/ using document root <none> and password file <none>
I20260807 01:46:21.336005 24592 fs_manager.cc:362] Metadata directory not provided
I20260807 01:46:21.336187 24592 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 01:46:21.336618 24592 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 01:46:21.337852 24592 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-0-root/instance:
uuid: "3262a9d183914b699227fdba0ca5bed3"
format_stamp: "Formatted at 2026-08-07 01:46:21 on dist-test-slave-zj8q"
I20260807 01:46:21.343910 24592 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 01:46:21.347762 27061 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 01:46:21.348590 24592 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260807 01:46:21.348915 24592 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-0-root
uuid: "3262a9d183914b699227fdba0ca5bed3"
format_stamp: "Formatted at 2026-08-07 01:46:21 on dist-test-slave-zj8q"
I20260807 01:46:21.349213 24592 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 01:46:21.364820 24592 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 01:46:21.366052 24592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 01:46:21.367654 24592 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 01:46:21.370330 24592 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 01:46:21.370543 24592 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 01:46:21.370790 24592 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 01:46:21.370937 24592 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 01:46:21.418725 24592 rpc_server.cc:307] RPC server started. Bound to: 127.24.4.1:35499
I20260807 01:46:21.419021 27146 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.4.1:35499 every 8 connection(s)
I20260807 01:46:21.423732 24592 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 01:46:21.432996 27153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 01:46:21.431752 27152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:46:21.436309 24592 server_base.cc:1061] running on GCE node
W20260807 01:46:21.436806 27158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:46:21.437780 24592 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 01:46:21.438059 24592 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 01:46:21.438300 24592 hybrid_clock.cc:648] HybridClock initialized: now 1786067181438281 us; error 0 us; skew 500 ppm
I20260807 01:46:21.438807 27148 heartbeater.cc:344] Connected to a master server at 127.24.4.62:35633
I20260807 01:46:21.438992 24592 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 01:46:21.439177 27148 heartbeater.cc:461] Registering TS with master...
I20260807 01:46:21.440026 27148 heartbeater.cc:507] Master 127.24.4.62:35633 requested a full tablet report, sending...
I20260807 01:46:21.441763 24592 webserver.cc:533] Webserver started at http://127.24.4.2:33263/ using document root <none> and password file <none>
I20260807 01:46:21.442413 24592 fs_manager.cc:362] Metadata directory not provided
I20260807 01:46:21.442622 24592 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 01:46:21.442655 26954 ts_manager.cc:194] Registered new tserver with Master: 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:46:21.442890 24592 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 01:46:21.444319 24592 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-1-root/instance:
uuid: "3478729505b046c2a0051c2d0130873f"
format_stamp: "Formatted at 2026-08-07 01:46:21 on dist-test-slave-zj8q"
I20260807 01:46:21.444628 26954 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46134
I20260807 01:46:21.449276 24592 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 01:46:21.453083 27167 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 01:46:21.453908 24592 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 01:46:21.454279 24592 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-1-root
uuid: "3478729505b046c2a0051c2d0130873f"
format_stamp: "Formatted at 2026-08-07 01:46:21 on dist-test-slave-zj8q"
I20260807 01:46:21.454576 24592 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 01:46:21.466280 24592 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 01:46:21.467494 24592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 01:46:21.469045 24592 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 01:46:21.471484 24592 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 01:46:21.471709 24592 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 01:46:21.471938 24592 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 01:46:21.472088 24592 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 01:46:21.515453 24592 rpc_server.cc:307] RPC server started. Bound to: 127.24.4.2:34855
I20260807 01:46:21.515575 27269 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.4.2:34855 every 8 connection(s)
I20260807 01:46:21.520334 24592 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 01:46:21.528291 27275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 01:46:21.530076 27277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 01:46:21.533771 27279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:46:21.534307 24592 server_base.cc:1061] running on GCE node
I20260807 01:46:21.534334 27270 heartbeater.cc:344] Connected to a master server at 127.24.4.62:35633
I20260807 01:46:21.534986 27270 heartbeater.cc:461] Registering TS with master...
I20260807 01:46:21.535445 24592 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 01:46:21.535665 24592 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 01:46:21.535825 24592 hybrid_clock.cc:648] HybridClock initialized: now 1786067181535808 us; error 0 us; skew 500 ppm
I20260807 01:46:21.535789 27270 heartbeater.cc:507] Master 127.24.4.62:35633 requested a full tablet report, sending...
I20260807 01:46:21.536506 24592 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 01:46:21.538105 26954 ts_manager.cc:194] Registered new tserver with Master: 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:21.539479 24592 webserver.cc:533] Webserver started at http://127.24.4.3:37519/ using document root <none> and password file <none>
I20260807 01:46:21.539721 26954 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46138
I20260807 01:46:21.540046 24592 fs_manager.cc:362] Metadata directory not provided
I20260807 01:46:21.540279 24592 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 01:46:21.540527 24592 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 01:46:21.541950 24592 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-2-root/instance:
uuid: "fcf4e45defb94276bcfa68392711dade"
format_stamp: "Formatted at 2026-08-07 01:46:21 on dist-test-slave-zj8q"
I20260807 01:46:21.547506 24592 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 01:46:21.551331 27285 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 01:46:21.552372 24592 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 01:46:21.552667 24592 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-2-root
uuid: "fcf4e45defb94276bcfa68392711dade"
format_stamp: "Formatted at 2026-08-07 01:46:21 on dist-test-slave-zj8q"
I20260807 01:46:21.552958 24592 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 01:46:21.567981 24592 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 01:46:21.569100 24592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 01:46:21.570670 24592 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 01:46:21.573031 24592 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 01:46:21.573244 24592 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 01:46:21.573485 24592 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 01:46:21.573660 24592 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 01:46:21.616885 24592 rpc_server.cc:307] RPC server started. Bound to: 127.24.4.3:42429
I20260807 01:46:21.617350 27378 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.4.3:42429 every 8 connection(s)
I20260807 01:46:21.632634 27379 heartbeater.cc:344] Connected to a master server at 127.24.4.62:35633
I20260807 01:46:21.633065 27379 heartbeater.cc:461] Registering TS with master...
I20260807 01:46:21.633838 27379 heartbeater.cc:507] Master 127.24.4.62:35633 requested a full tablet report, sending...
I20260807 01:46:21.636372 26954 ts_manager.cc:194] Registered new tserver with Master: fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:21.637178 24592 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017121134s
I20260807 01:46:21.638199 26954 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46152
I20260807 01:46:21.666676 26954 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46162:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 01:46:21.672700 26954 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 01:46:21.891096 27211 tablet_service.cc:1511] Processing CreateTablet for tablet cf4d8c8825e149adbd4565e0c8fdd109 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260807 01:46:21.892788 27211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf4d8c8825e149adbd4565e0c8fdd109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:21.893253 27205 tablet_service.cc:1511] Processing CreateTablet for tablet 51cef821f0284d8e9369e98d38f80e73 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260807 01:46:21.895361 27204 tablet_service.cc:1511] Processing CreateTablet for tablet 37478d818c864abc83662e98299718d1 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260807 01:46:21.895309 27203 tablet_service.cc:1511] Processing CreateTablet for tablet 09799289505a4c188931e0f3ed0adfb3 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260807 01:46:21.896369 27209 tablet_service.cc:1511] Processing CreateTablet for tablet 73e9ae0228aa41a996315efe07c04299 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260807 01:46:21.895311 27210 tablet_service.cc:1511] Processing CreateTablet for tablet c4c4340b503246eca370314d2781034a (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260807 01:46:21.898667 27206 tablet_service.cc:1511] Processing CreateTablet for tablet 8767bf1fc98c4effa8f3b5f5113afe40 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260807 01:46:21.899540 27201 tablet_service.cc:1511] Processing CreateTablet for tablet 3806d33df8144ca5a662db5461461b43 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260807 01:46:21.900466 27205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51cef821f0284d8e9369e98d38f80e73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:21.901139 27201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3806d33df8144ca5a662db5461461b43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:21.903151 27204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37478d818c864abc83662e98299718d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:21.904269 27199 tablet_service.cc:1511] Processing CreateTablet for tablet 8b00b70a17a64e288e95507deb091fc1 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260807 01:46:21.904781 27206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8767bf1fc98c4effa8f3b5f5113afe40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:21.905588 27197 tablet_service.cc:1511] Processing CreateTablet for tablet 94962ef95ca14973bc39a17ed76c0087 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260807 01:46:21.906687 27210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4c4340b503246eca370314d2781034a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:21.909584 27203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09799289505a4c188931e0f3ed0adfb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:21.914883 27209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73e9ae0228aa41a996315efe07c04299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:21.916491 27199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b00b70a17a64e288e95507deb091fc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:21.917225 27197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94962ef95ca14973bc39a17ed76c0087. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:21.934715 27317 tablet_service.cc:1511] Processing CreateTablet for tablet 51cef821f0284d8e9369e98d38f80e73 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260807 01:46:21.935197 27324 tablet_service.cc:1511] Processing CreateTablet for tablet cf4d8c8825e149adbd4565e0c8fdd109 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260807 01:46:21.936791 27317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51cef821f0284d8e9369e98d38f80e73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:21.940560 27319 tablet_service.cc:1511] Processing CreateTablet for tablet 8767bf1fc98c4effa8f3b5f5113afe40 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260807 01:46:21.941224 27322 tablet_service.cc:1511] Processing CreateTablet for tablet c4c4340b503246eca370314d2781034a (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260807 01:46:21.940560 27320 tablet_service.cc:1511] Processing CreateTablet for tablet 73e9ae0228aa41a996315efe07c04299 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260807 01:46:21.942842 27324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf4d8c8825e149adbd4565e0c8fdd109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:21.943573 27319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8767bf1fc98c4effa8f3b5f5113afe40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:21.944281 27322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4c4340b503246eca370314d2781034a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:21.945070 27320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73e9ae0228aa41a996315efe07c04299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:21.946679 27313 tablet_service.cc:1511] Processing CreateTablet for tablet 09799289505a4c188931e0f3ed0adfb3 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260807 01:46:21.948170 27313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09799289505a4c188931e0f3ed0adfb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:21.951416 27312 tablet_service.cc:1511] Processing CreateTablet for tablet 3806d33df8144ca5a662db5461461b43 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260807 01:46:21.940515 27314 tablet_service.cc:1511] Processing CreateTablet for tablet 37478d818c864abc83662e98299718d1 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260807 01:46:21.952973 27312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3806d33df8144ca5a662db5461461b43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:21.953711 27314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37478d818c864abc83662e98299718d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:21.955727 27310 tablet_service.cc:1511] Processing CreateTablet for tablet 8b00b70a17a64e288e95507deb091fc1 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260807 01:46:21.957254 27310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b00b70a17a64e288e95507deb091fc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:21.959484 27309 tablet_service.cc:1511] Processing CreateTablet for tablet 94962ef95ca14973bc39a17ed76c0087 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260807 01:46:21.960928 27309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94962ef95ca14973bc39a17ed76c0087. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.018370 27322 tablet_service.cc:1511] Processing CreateTablet for tablet 30561911ad3945159907e917cc1bafa7 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260807 01:46:22.019893 27322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30561911ad3945159907e917cc1bafa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.038535 27201 tablet_service.cc:1511] Processing CreateTablet for tablet 30561911ad3945159907e917cc1bafa7 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260807 01:46:22.040083 27201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30561911ad3945159907e917cc1bafa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.042126 27322 tablet_service.cc:1511] Processing CreateTablet for tablet 08edf043c99f445b99e723211e8e0ad0 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260807 01:46:22.043852 27322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08edf043c99f445b99e723211e8e0ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.066413 27405 tablet_bootstrap.cc:492] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:22.071519 27210 tablet_service.cc:1511] Processing CreateTablet for tablet 498dfe25861a4b4fbf2aca930b357274 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260807 01:46:22.073078 27210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 498dfe25861a4b4fbf2aca930b357274. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.073104 27205 tablet_service.cc:1511] Processing CreateTablet for tablet f7a8b225f3fd4b65a1f62d70c6fd2bb7 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260807 01:46:22.075438 27405 tablet_bootstrap.cc:654] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:22.076330 27313 tablet_service.cc:1511] Processing CreateTablet for tablet f7a8b225f3fd4b65a1f62d70c6fd2bb7 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260807 01:46:22.077809 27313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7a8b225f3fd4b65a1f62d70c6fd2bb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.087643 27199 tablet_service.cc:1511] Processing CreateTablet for tablet b3a9345c5a2946019985c5a10aacd64f (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260807 01:46:22.090185 27324 tablet_service.cc:1511] Processing CreateTablet for tablet 498dfe25861a4b4fbf2aca930b357274 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260807 01:46:22.090844 27209 tablet_service.cc:1511] Processing CreateTablet for tablet de3b20c9b4a7467ea9e6c098167df7e3 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260807 01:46:22.091607 27324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 498dfe25861a4b4fbf2aca930b357274. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.095572 27314 tablet_service.cc:1511] Processing CreateTablet for tablet b3a9345c5a2946019985c5a10aacd64f (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260807 01:46:22.097008 27314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3a9345c5a2946019985c5a10aacd64f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.098652 27094 tablet_service.cc:1511] Processing CreateTablet for tablet c4c4340b503246eca370314d2781034a (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260807 01:46:22.098992 27093 tablet_service.cc:1511] Processing CreateTablet for tablet 73e9ae0228aa41a996315efe07c04299 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260807 01:46:22.100158 27094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4c4340b503246eca370314d2781034a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.101320 27098 tablet_service.cc:1511] Processing CreateTablet for tablet cf4d8c8825e149adbd4565e0c8fdd109 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260807 01:46:22.101707 27092 tablet_service.cc:1511] Processing CreateTablet for tablet 8767bf1fc98c4effa8f3b5f5113afe40 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260807 01:46:22.104243 27090 tablet_service.cc:1511] Processing CreateTablet for tablet 37478d818c864abc83662e98299718d1 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260807 01:46:22.104777 27092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8767bf1fc98c4effa8f3b5f5113afe40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.105768 27090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37478d818c864abc83662e98299718d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.106472 27093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73e9ae0228aa41a996315efe07c04299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.107389 27089 tablet_service.cc:1511] Processing CreateTablet for tablet 09799289505a4c188931e0f3ed0adfb3 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260807 01:46:22.108880 27089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09799289505a4c188931e0f3ed0adfb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.111423 27088 tablet_service.cc:1511] Processing CreateTablet for tablet 3806d33df8144ca5a662db5461461b43 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260807 01:46:22.112983 27088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3806d33df8144ca5a662db5461461b43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.114789 27205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7a8b225f3fd4b65a1f62d70c6fd2bb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.115455 27209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de3b20c9b4a7467ea9e6c098167df7e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.115900 27087 tablet_service.cc:1511] Processing CreateTablet for tablet 8b00b70a17a64e288e95507deb091fc1 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260807 01:46:22.117404 27087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b00b70a17a64e288e95507deb091fc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.118768 27091 tablet_service.cc:1511] Processing CreateTablet for tablet 51cef821f0284d8e9369e98d38f80e73 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260807 01:46:22.120720 27091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51cef821f0284d8e9369e98d38f80e73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.119799 27086 tablet_service.cc:1511] Processing CreateTablet for tablet 94962ef95ca14973bc39a17ed76c0087 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260807 01:46:22.123680 27086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94962ef95ca14973bc39a17ed76c0087. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.126955 27098 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf4d8c8825e149adbd4565e0c8fdd109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.130215 27317 tablet_service.cc:1511] Processing CreateTablet for tablet de3b20c9b4a7467ea9e6c098167df7e3 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260807 01:46:22.139752 27404 tablet_bootstrap.cc:492] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:22.143791 27319 tablet_service.cc:1511] Processing CreateTablet for tablet 270fd0241ee84fcea8889fff6d973747 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260807 01:46:22.151676 27204 tablet_service.cc:1511] Processing CreateTablet for tablet 05e95c9ad127429dbdd5252c0670f22f (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260807 01:46:22.142037 27317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de3b20c9b4a7467ea9e6c098167df7e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.141815 27310 tablet_service.cc:1511] Processing CreateTablet for tablet 2b4fd523ba5d463b8508a943e293612c (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260807 01:46:22.153529 27309 tablet_service.cc:1511] Processing CreateTablet for tablet 1688e72d717b430e86793ecd17b2e952 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260807 01:46:22.145984 27206 tablet_service.cc:1511] Processing CreateTablet for tablet 08edf043c99f445b99e723211e8e0ad0 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260807 01:46:22.160519 27310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b4fd523ba5d463b8508a943e293612c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.174304 27199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3a9345c5a2946019985c5a10aacd64f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.178830 27320 tablet_service.cc:1511] Processing CreateTablet for tablet 05e95c9ad127429dbdd5252c0670f22f (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260807 01:46:22.193537 27094 tablet_service.cc:1511] Processing CreateTablet for tablet 53d8de481420436498aadc9930032da5 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260807 01:46:22.194370 27319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 270fd0241ee84fcea8889fff6d973747. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.195061 27094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53d8de481420436498aadc9930032da5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.195502 27404 tablet_bootstrap.cc:654] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:22.200855 27203 tablet_service.cc:1511] Processing CreateTablet for tablet 2b4fd523ba5d463b8508a943e293612c (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260807 01:46:22.189047 27204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05e95c9ad127429dbdd5252c0670f22f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.208511 27206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08edf043c99f445b99e723211e8e0ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.208683 27405 tablet_bootstrap.cc:492] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:22.210528 27309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1688e72d717b430e86793ecd17b2e952. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.210799 27405 ts_tablet_manager.cc:1403] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.145s user 0.025s sys 0.024s
I20260807 01:46:22.212321 27086 tablet_service.cc:1511] Processing CreateTablet for tablet 08edf043c99f445b99e723211e8e0ad0 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260807 01:46:22.213410 27320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05e95c9ad127429dbdd5252c0670f22f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.218535 27203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b4fd523ba5d463b8508a943e293612c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.219242 27405 raft_consensus.cc:359] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:22.219996 27405 raft_consensus.cc:385] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:22.220291 27405 raft_consensus.cc:740] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:22.218426 27086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08edf043c99f445b99e723211e8e0ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.226711 27407 tablet_bootstrap.cc:492] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:22.241654 27405 consensus_queue.cc:260] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:22.248170 27036 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 01:46:22.248077 27092 tablet_service.cc:1511] Processing CreateTablet for tablet f7a8b225f3fd4b65a1f62d70c6fd2bb7 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260807 01:46:22.252223 27211 tablet_service.cc:1511] Processing CreateTablet for tablet 1688e72d717b430e86793ecd17b2e952 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260807 01:46:22.253672 27211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1688e72d717b430e86793ecd17b2e952. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.255554 27197 tablet_service.cc:1511] Processing CreateTablet for tablet 53d8de481420436498aadc9930032da5 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260807 01:46:22.257284 27197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53d8de481420436498aadc9930032da5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.259737 27201 tablet_service.cc:1511] Processing CreateTablet for tablet 270fd0241ee84fcea8889fff6d973747 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260807 01:46:22.264948 27314 tablet_service.cc:1511] Processing CreateTablet for tablet aff03a52fbfd498ab6d04a579e96fd0e (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260807 01:46:22.274977 27314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aff03a52fbfd498ab6d04a579e96fd0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.242594 27324 tablet_service.cc:1511] Processing CreateTablet for tablet 53d8de481420436498aadc9930032da5 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260807 01:46:22.277967 27324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53d8de481420436498aadc9930032da5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.268883 27322 tablet_service.cc:1511] Processing CreateTablet for tablet 8dd88b3a4c37491bbe304dae1bdb08fb (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260807 01:46:22.283051 27322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dd88b3a4c37491bbe304dae1bdb08fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.284042 27092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7a8b225f3fd4b65a1f62d70c6fd2bb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.285600 27087 tablet_service.cc:1511] Processing CreateTablet for tablet 498dfe25861a4b4fbf2aca930b357274 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260807 01:46:22.286968 27087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 498dfe25861a4b4fbf2aca930b357274. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.296145 27201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 270fd0241ee84fcea8889fff6d973747. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.300844 27317 tablet_service.cc:1511] Processing CreateTablet for tablet 21e1a5771ca04812aac6f5abed966ccd (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260807 01:46:22.302544 27317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21e1a5771ca04812aac6f5abed966ccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.303884 27405 ts_tablet_manager.cc:1434] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.093s user 0.007s sys 0.006s
I20260807 01:46:22.305683 27098 tablet_service.cc:1511] Processing CreateTablet for tablet b3a9345c5a2946019985c5a10aacd64f (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260807 01:46:22.307741 27205 tablet_service.cc:1511] Processing CreateTablet for tablet a0e280c976844349b4874a479aa08de2 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260807 01:46:22.308439 27210 tablet_service.cc:1511] Processing CreateTablet for tablet 8dd88b3a4c37491bbe304dae1bdb08fb (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260807 01:46:22.311431 27404 tablet_bootstrap.cc:492] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:22.260252 27312 tablet_service.cc:1511] Processing CreateTablet for tablet a0e280c976844349b4874a479aa08de2 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260807 01:46:22.324316 27404 ts_tablet_manager.cc:1403] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.185s user 0.010s sys 0.014s
I20260807 01:46:22.324882 27312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0e280c976844349b4874a479aa08de2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.307741 27203 tablet_service.cc:1511] Processing CreateTablet for tablet aff03a52fbfd498ab6d04a579e96fd0e (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260807 01:46:22.315030 27091 tablet_service.cc:1511] Processing CreateTablet for tablet 30561911ad3945159907e917cc1bafa7 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260807 01:46:22.326447 27270 heartbeater.cc:499] Master 127.24.4.62:35633 was elected leader, sending a full tablet report...
I20260807 01:46:22.331020 27205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0e280c976844349b4874a479aa08de2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.332383 27404 raft_consensus.cc:359] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:22.333155 27404 raft_consensus.cc:385] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:22.333441 27404 raft_consensus.cc:740] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:22.344789 27310 tablet_service.cc:1511] Processing CreateTablet for tablet 31708844dc4f41feb6a7a2820cfc0e0b (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260807 01:46:22.354279 27091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30561911ad3945159907e917cc1bafa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.351529 27407 tablet_bootstrap.cc:654] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:22.362087 27098 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3a9345c5a2946019985c5a10aacd64f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.362576 27405 tablet_bootstrap.cc:492] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:22.365070 27090 tablet_service.cc:1511] Processing CreateTablet for tablet de3b20c9b4a7467ea9e6c098167df7e3 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260807 01:46:22.366905 27310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31708844dc4f41feb6a7a2820cfc0e0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.368396 27088 tablet_service.cc:1511] Processing CreateTablet for tablet 05e95c9ad127429dbdd5252c0670f22f (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260807 01:46:22.358533 27404 consensus_queue.cc:260] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:22.370690 27320 tablet_service.cc:1511] Processing CreateTablet for tablet 67fed460b08e4124a6d4484db52478b1 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260807 01:46:22.402237 27324 tablet_service.cc:1511] Processing CreateTablet for tablet 9db18f8163fc4d79aceac26f0e762e2b (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260807 01:46:22.407447 27203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aff03a52fbfd498ab6d04a579e96fd0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.380714 27319 tablet_service.cc:1511] Processing CreateTablet for tablet 1c4e145f8f52431fa41b1f596e2f1133 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260807 01:46:22.411036 27320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67fed460b08e4124a6d4484db52478b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.402820 27405 tablet_bootstrap.cc:654] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:22.422276 27309 tablet_service.cc:1511] Processing CreateTablet for tablet c8799906440c4ef19cfbadc18cf32ef0 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260807 01:46:22.424647 27087 tablet_service.cc:1511] Processing CreateTablet for tablet 1688e72d717b430e86793ecd17b2e952 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260807 01:46:22.427977 27209 tablet_service.cc:1511] Processing CreateTablet for tablet 21e1a5771ca04812aac6f5abed966ccd (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260807 01:46:22.349907 27313 tablet_service.cc:1511] Processing CreateTablet for tablet 3b3fc468354b48cf80eee526def18e6c (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260807 01:46:22.441684 27319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c4e145f8f52431fa41b1f596e2f1133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.448527 27090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de3b20c9b4a7467ea9e6c098167df7e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.450882 27148 heartbeater.cc:499] Master 127.24.4.62:35633 was elected leader, sending a full tablet report...
I20260807 01:46:22.398370 27089 tablet_service.cc:1511] Processing CreateTablet for tablet 2b4fd523ba5d463b8508a943e293612c (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260807 01:46:22.458287 27089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b4fd523ba5d463b8508a943e293612c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.463264 27088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05e95c9ad127429dbdd5252c0670f22f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.467067 27087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1688e72d717b430e86793ecd17b2e952. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.468150 27094 tablet_service.cc:1511] Processing CreateTablet for tablet a0e280c976844349b4874a479aa08de2 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260807 01:46:22.469547 27094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0e280c976844349b4874a479aa08de2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.473810 27210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dd88b3a4c37491bbe304dae1bdb08fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.475392 27201 tablet_service.cc:1511] Processing CreateTablet for tablet 31708844dc4f41feb6a7a2820cfc0e0b (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260807 01:46:22.476729 27201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31708844dc4f41feb6a7a2820cfc0e0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.486847 27209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21e1a5771ca04812aac6f5abed966ccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.488610 27199 tablet_service.cc:1511] Processing CreateTablet for tablet 3b3fc468354b48cf80eee526def18e6c (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260807 01:46:22.493569 27204 tablet_service.cc:1511] Processing CreateTablet for tablet 67fed460b08e4124a6d4484db52478b1 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260807 01:46:22.495905 27404 ts_tablet_manager.cc:1434] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.171s user 0.024s sys 0.012s
I20260807 01:46:22.496796 27379 heartbeater.cc:499] Master 127.24.4.62:35633 was elected leader, sending a full tablet report...
I20260807 01:46:22.502836 27404 tablet_bootstrap.cc:492] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:22.506258 27324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9db18f8163fc4d79aceac26f0e762e2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.509473 27205 tablet_service.cc:1511] Processing CreateTablet for tablet 9db18f8163fc4d79aceac26f0e762e2b (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260807 01:46:22.514514 27309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8799906440c4ef19cfbadc18cf32ef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.527805 27407 tablet_bootstrap.cc:492] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:22.528430 27407 ts_tablet_manager.cc:1403] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.302s user 0.025s sys 0.022s
I20260807 01:46:22.529893 27197 tablet_service.cc:1511] Processing CreateTablet for tablet b2dac5b83a404fbba1506670f8b8b45f (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260807 01:46:22.535645 27211 tablet_service.cc:1511] Processing CreateTablet for tablet 965af9378c6a4617b1d50f229ffc2c5b (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260807 01:46:22.537191 27093 tablet_service.cc:1511] Processing CreateTablet for tablet 270fd0241ee84fcea8889fff6d973747 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260807 01:46:22.538383 27093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 270fd0241ee84fcea8889fff6d973747. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.517534 27086 tablet_service.cc:1511] Processing CreateTablet for tablet aff03a52fbfd498ab6d04a579e96fd0e (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260807 01:46:22.542246 27086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aff03a52fbfd498ab6d04a579e96fd0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.542588 27313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b3fc468354b48cf80eee526def18e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.542778 27404 tablet_bootstrap.cc:654] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:22.547647 27087 tablet_service.cc:1511] Processing CreateTablet for tablet 21e1a5771ca04812aac6f5abed966ccd (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260807 01:46:22.549338 27312 tablet_service.cc:1511] Processing CreateTablet for tablet 965af9378c6a4617b1d50f229ffc2c5b (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260807 01:46:22.559983 27312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 965af9378c6a4617b1d50f229ffc2c5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.560743 27087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21e1a5771ca04812aac6f5abed966ccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.565050 27098 tablet_service.cc:1511] Processing CreateTablet for tablet 3b3fc468354b48cf80eee526def18e6c (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260807 01:46:22.555701 27407 raft_consensus.cc:359] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:22.566045 27407 raft_consensus.cc:385] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:22.566468 27407 raft_consensus.cc:740] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:22.567416 27407 consensus_queue.cc:260] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:22.540685 27092 tablet_service.cc:1511] Processing CreateTablet for tablet 8dd88b3a4c37491bbe304dae1bdb08fb (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260807 01:46:22.557300 27201 tablet_service.cc:1511] Processing CreateTablet for tablet c8799906440c4ef19cfbadc18cf32ef0 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260807 01:46:22.567986 27409 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:22.490073 27199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b3fc468354b48cf80eee526def18e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.571017 27098 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b3fc468354b48cf80eee526def18e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.581921 27409 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:22.589781 27409 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:22.576880 27091 tablet_service.cc:1511] Processing CreateTablet for tablet 31708844dc4f41feb6a7a2820cfc0e0b (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260807 01:46:22.612499 27204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67fed460b08e4124a6d4484db52478b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.612677 27092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dd88b3a4c37491bbe304dae1bdb08fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.626945 27322 tablet_service.cc:1511] Processing CreateTablet for tablet d235fc7e09b8465f8ddc570370605464 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260807 01:46:22.653273 27322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d235fc7e09b8465f8ddc570370605464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.659798 27407 ts_tablet_manager.cc:1434] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.131s user 0.010s sys 0.005s
I20260807 01:46:22.660933 27206 tablet_service.cc:1511] Processing CreateTablet for tablet d235fc7e09b8465f8ddc570370605464 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260807 01:46:22.676568 27314 tablet_service.cc:1511] Processing CreateTablet for tablet 60a03ab231464be89d6cce9e6988cfba (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260807 01:46:22.678347 27205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9db18f8163fc4d79aceac26f0e762e2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.704288 27407 tablet_bootstrap.cc:492] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:22.678125 27314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60a03ab231464be89d6cce9e6988cfba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.696156 27404 tablet_bootstrap.cc:492] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:22.681769 27405 tablet_bootstrap.cc:492] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:22.706651 27091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31708844dc4f41feb6a7a2820cfc0e0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.702421 27414 raft_consensus.cc:493] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:22.721101 27206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d235fc7e09b8465f8ddc570370605464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.721839 27404 ts_tablet_manager.cc:1403] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.219s user 0.011s sys 0.012s
I20260807 01:46:22.722626 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:22.723934 27414 raft_consensus.cc:515] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:22.725035 27201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8799906440c4ef19cfbadc18cf32ef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.725358 27405 ts_tablet_manager.cc:1403] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.363s user 0.018s sys 0.017s
W20260807 01:46:22.727574 27172 leader_election.cc:343] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:22.728925 27405 raft_consensus.cc:359] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:22.729676 27405 raft_consensus.cc:385] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:22.730057 27211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 965af9378c6a4617b1d50f229ffc2c5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.732328 27090 tablet_service.cc:1511] Processing CreateTablet for tablet 9db18f8163fc4d79aceac26f0e762e2b (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260807 01:46:22.730029 27414 leader_election.cc:290] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:22.734910 27197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2dac5b83a404fbba1506670f8b8b45f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.684897 27317 tablet_service.cc:1511] Processing CreateTablet for tablet 565d0a4463ac45e4bad1886cc0095ff7 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260807 01:46:22.725149 27404 raft_consensus.cc:359] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:22.736691 27404 raft_consensus.cc:385] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:22.737015 27404 raft_consensus.cc:740] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:22.737913 27404 consensus_queue.cc:260] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:22.740204 27203 tablet_service.cc:1511] Processing CreateTablet for tablet 60a03ab231464be89d6cce9e6988cfba (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260807 01:46:22.741663 27203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60a03ab231464be89d6cce9e6988cfba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.742208 27324 tablet_service.cc:1511] Processing CreateTablet for tablet d8e18e65b8d2418cb159040f0fae8be3 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260807 01:46:22.750597 27090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9db18f8163fc4d79aceac26f0e762e2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.702132 27210 tablet_service.cc:1511] Processing CreateTablet for tablet d8e18e65b8d2418cb159040f0fae8be3 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260807 01:46:22.753850 27210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8e18e65b8d2418cb159040f0fae8be3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.729969 27405 raft_consensus.cc:740] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:22.759266 27407 tablet_bootstrap.cc:654] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:22.759341 27098 tablet_service.cc:1511] Processing CreateTablet for tablet c8799906440c4ef19cfbadc18cf32ef0 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260807 01:46:22.740357 27404 ts_tablet_manager.cc:1434] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.018s user 0.007s sys 0.000s
I20260807 01:46:22.768829 27309 tablet_service.cc:1511] Processing CreateTablet for tablet b3ee061f38f84a71ad3991835a2e56dc (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260807 01:46:22.770830 27404 tablet_bootstrap.cc:492] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:22.757587 27405 consensus_queue.cc:260] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:22.778398 27088 tablet_service.cc:1511] Processing CreateTablet for tablet 965af9378c6a4617b1d50f229ffc2c5b (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260807 01:46:22.779984 27094 tablet_service.cc:1511] Processing CreateTablet for tablet 67fed460b08e4124a6d4484db52478b1 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260807 01:46:22.782498 27405 ts_tablet_manager.cc:1434] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.057s user 0.008s sys 0.000s
I20260807 01:46:22.809327 27405 tablet_bootstrap.cc:492] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:22.791115 27209 tablet_service.cc:1511] Processing CreateTablet for tablet b3ee061f38f84a71ad3991835a2e56dc (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260807 01:46:22.794973 27320 tablet_service.cc:1511] Processing CreateTablet for tablet ed1089e78f8246fcb8af73c74af99b19 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260807 01:46:22.776358 27324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8e18e65b8d2418cb159040f0fae8be3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.797477 27313 tablet_service.cc:1511] Processing CreateTablet for tablet 81439f7f101341049bd086b353ca3495 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260807 01:46:22.817180 27313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81439f7f101341049bd086b353ca3495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.817333 27089 tablet_service.cc:1511] Processing CreateTablet for tablet d235fc7e09b8465f8ddc570370605464 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260807 01:46:22.798929 27310 tablet_service.cc:1511] Processing CreateTablet for tablet b2dac5b83a404fbba1506670f8b8b45f (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260807 01:46:22.800490 27093 tablet_service.cc:1511] Processing CreateTablet for tablet d53e096d3ad04a5bb333660000313179 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260807 01:46:22.801379 27319 tablet_service.cc:1511] Processing CreateTablet for tablet d53e096d3ad04a5bb333660000313179 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260807 01:46:22.818478 27317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 565d0a4463ac45e4bad1886cc0095ff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.819305 27319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d53e096d3ad04a5bb333660000313179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.802929 27199 tablet_service.cc:1511] Processing CreateTablet for tablet 565d0a4463ac45e4bad1886cc0095ff7 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260807 01:46:22.807355 27211 tablet_service.cc:1511] Processing CreateTablet for tablet 1c4e145f8f52431fa41b1f596e2f1133 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260807 01:46:22.783769 27098 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8799906440c4ef19cfbadc18cf32ef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.825927 27199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 565d0a4463ac45e4bad1886cc0095ff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.834563 27088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 965af9378c6a4617b1d50f229ffc2c5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.837656 27094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67fed460b08e4124a6d4484db52478b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.839568 27093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d53e096d3ad04a5bb333660000313179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.844568 27086 tablet_service.cc:1511] Processing CreateTablet for tablet d8e18e65b8d2418cb159040f0fae8be3 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260807 01:46:22.848678 27420 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:22.849238 27420 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:22.832427 27404 tablet_bootstrap.cc:654] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:22.846058 27086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8e18e65b8d2418cb159040f0fae8be3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.858206 27211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c4e145f8f52431fa41b1f596e2f1133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.861272 27309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3ee061f38f84a71ad3991835a2e56dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.876031 27420 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:22.883095 27209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3ee061f38f84a71ad3991835a2e56dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.896430 27310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2dac5b83a404fbba1506670f8b8b45f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260807 01:46:22.896740 26931 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 57472.1us
I20260807 01:46:22.898542 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:22.858384 27206 tablet_service.cc:1511] Processing CreateTablet for tablet ed1089e78f8246fcb8af73c74af99b19 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260807 01:46:22.905421 27201 tablet_service.cc:1511] Processing CreateTablet for tablet d53e096d3ad04a5bb333660000313179 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260807 01:46:22.915990 27205 tablet_service.cc:1511] Processing CreateTablet for tablet 6420178dae2448f78216552b1b03c6e6 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260807 01:46:22.928241 27405 tablet_bootstrap.cc:654] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:22.931835 27204 tablet_service.cc:1511] Processing CreateTablet for tablet 54eb75662df043488517f09bff1dfcff (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260807 01:46:22.933813 27092 tablet_service.cc:1511] Processing CreateTablet for tablet 60a03ab231464be89d6cce9e6988cfba (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260807 01:46:22.934505 27409 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:22.867125 27210 tablet_service.cc:1511] Processing CreateTablet for tablet 81439f7f101341049bd086b353ca3495 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260807 01:46:22.951150 27197 tablet_service.cc:1511] Processing CreateTablet for tablet b510849c3131475d8c1e032378dc893b (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260807 01:46:22.894811 27089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d235fc7e09b8465f8ddc570370605464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.971613 27320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed1089e78f8246fcb8af73c74af99b19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.985316 27206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed1089e78f8246fcb8af73c74af99b19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:22.991748 27319 tablet_service.cc:1511] Processing CreateTablet for tablet b510849c3131475d8c1e032378dc893b (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260807 01:46:22.992790 27312 tablet_service.cc:1511] Processing CreateTablet for tablet 6420178dae2448f78216552b1b03c6e6 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260807 01:46:22.994575 27404 tablet_bootstrap.cc:492] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:22.995076 27404 ts_tablet_manager.cc:1403] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.224s user 0.019s sys 0.000s
I20260807 01:46:23.001472 27407 tablet_bootstrap.cc:492] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:22.964125 27338 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:22.985152 27091 tablet_service.cc:1511] Processing CreateTablet for tablet 565d0a4463ac45e4bad1886cc0095ff7 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260807 01:46:23.002008 27407 ts_tablet_manager.cc:1403] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.298s user 0.010s sys 0.019s
I20260807 01:46:22.972353 27409 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.006078 27170 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:23.003319 27404 raft_consensus.cc:359] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.004292 27312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6420178dae2448f78216552b1b03c6e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.011507 27210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81439f7f101341049bd086b353ca3495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.011731 27409 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:23.014277 27092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60a03ab231464be89d6cce9e6988cfba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.024400 27409 raft_consensus.cc:2804] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:23.024791 27409 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:23.014650 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:23.025149 27409 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:23.015600 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:23.020397 27404 raft_consensus.cc:385] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:23.026638 27404 raft_consensus.cc:740] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
W20260807 01:46:23.027318 27172 leader_election.cc:343] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:23.024940 27091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 565d0a4463ac45e4bad1886cc0095ff7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260807 01:46:23.026681 27287 leader_election.cc:343] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:23.029139 27404 consensus_queue.cc:260] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.023888 27205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6420178dae2448f78216552b1b03c6e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.035385 27322 tablet_service.cc:1511] Processing CreateTablet for tablet b18526f467154c49b830de9b1b03066f (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260807 01:46:23.035969 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:23.037848 27287 leader_election.cc:343] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:23.038534 27201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d53e096d3ad04a5bb333660000313179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.038435 27287 leader_election.cc:304] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:23.039388 27414 raft_consensus.cc:2749] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:23.037847 27404 ts_tablet_manager.cc:1434] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.038s user 0.006s sys 0.001s
I20260807 01:46:23.042201 27404 tablet_bootstrap.cc:492] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:23.034638 27319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b510849c3131475d8c1e032378dc893b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.056154 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:23.056697 27088 tablet_service.cc:1511] Processing CreateTablet for tablet b3ee061f38f84a71ad3991835a2e56dc (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
W20260807 01:46:23.058236 27063 leader_election.cc:343] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:23.059621 27314 tablet_service.cc:1511] Processing CreateTablet for tablet 6b0f72aee1954e3ab89690c7cfb93677 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260807 01:46:23.060128 27317 tablet_service.cc:1511] Processing CreateTablet for tablet 54eb75662df043488517f09bff1dfcff (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260807 01:46:23.061892 27197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b510849c3131475d8c1e032378dc893b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.072338 27091 tablet_service.cc:1511] Processing CreateTablet for tablet 1c4e145f8f52431fa41b1f596e2f1133 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260807 01:46:23.058038 27088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3ee061f38f84a71ad3991835a2e56dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.013296 27407 raft_consensus.cc:359] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.080376 27407 raft_consensus.cc:385] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:23.080713 27407 raft_consensus.cc:740] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:23.080899 27087 tablet_service.cc:1511] Processing CreateTablet for tablet 81439f7f101341049bd086b353ca3495 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260807 01:46:23.079985 27204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54eb75662df043488517f09bff1dfcff. 1 dirs total, 0 dirs full, 0 dirs failed
W20260807 01:46:23.083838 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 78009.6us
I20260807 01:46:23.085739 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:23.088058 27090 tablet_service.cc:1511] Processing CreateTablet for tablet b2dac5b83a404fbba1506670f8b8b45f (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
W20260807 01:46:23.089078 27170 leader_election.cc:343] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:23.089821 27170 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:23.093262 27310 tablet_service.cc:1511] Processing CreateTablet for tablet bf07cc968f3343869d12c10e81ce75f9 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260807 01:46:23.074267 27098 tablet_service.cc:1511] Processing CreateTablet for tablet ed1089e78f8246fcb8af73c74af99b19 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260807 01:46:23.098384 27317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54eb75662df043488517f09bff1dfcff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.102468 27322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b18526f467154c49b830de9b1b03066f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.081542 27407 consensus_queue.cc:260] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.108095 27407 ts_tablet_manager.cc:1434] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.103s user 0.006s sys 0.000s
I20260807 01:46:23.111140 27407 tablet_bootstrap.cc:492] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:23.112648 27199 tablet_service.cc:1511] Processing CreateTablet for tablet b18526f467154c49b830de9b1b03066f (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260807 01:46:23.138361 27091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c4e145f8f52431fa41b1f596e2f1133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.142619 27409 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.144331 27438 raft_consensus.cc:2749] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:23.144992 27310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf07cc968f3343869d12c10e81ce75f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.147341 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:23.147996 27109 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:23.150568 27405 tablet_bootstrap.cc:492] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:23.151324 27405 ts_tablet_manager.cc:1403] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.342s user 0.031s sys 0.018s
I20260807 01:46:23.155694 27405 raft_consensus.cc:359] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.156903 27405 raft_consensus.cc:385] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:23.157217 27405 raft_consensus.cc:740] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:23.164455 27409 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:23.144654 27098 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed1089e78f8246fcb8af73c74af99b19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.166826 27087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81439f7f101341049bd086b353ca3495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.166491 27405 consensus_queue.cc:260] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.168795 27405 ts_tablet_manager.cc:1434] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.017s user 0.004s sys 0.003s
I20260807 01:46:23.172719 27405 tablet_bootstrap.cc:492] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:23.173301 27090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2dac5b83a404fbba1506670f8b8b45f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.177192 27199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b18526f467154c49b830de9b1b03066f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.183981 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:23.184669 27338 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:23.187361 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:23.189133 27064 leader_election.cc:343] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:23.189671 27064 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:23.191370 27322 tablet_service.cc:1511] Processing CreateTablet for tablet 879666f362ba4a2db3fdfe6738b37dec (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260807 01:46:23.191185 27420 raft_consensus.cc:2749] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:23.192806 27322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 879666f362ba4a2db3fdfe6738b37dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.189073 27109 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:23.202725 27324 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:59474 (request call id 32) took 1128 ms. Trace:
I20260807 01:46:23.203059 27313 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:59474 (request call id 38) took 1097 ms. Trace:
I20260807 01:46:23.204152 27314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b0f72aee1954e3ab89690c7cfb93677. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.205150 27172 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:23.206224 27409 raft_consensus.cc:2804] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:23.206780 27409 raft_consensus.cc:697] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:23.206437 27093 tablet_service.cc:1511] Processing CreateTablet for tablet b510849c3131475d8c1e032378dc893b (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260807 01:46:23.208050 27093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b510849c3131475d8c1e032378dc893b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.207692 27409 consensus_queue.cc:237] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.213680 27312 tablet_service.cc:1511] Processing CreateTablet for tablet 40f160d4dff84d8d9fb21b3489d01ff7 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260807 01:46:23.215636 27312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40f160d4dff84d8d9fb21b3489d01ff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.223773 27206 tablet_service.cc:1511] Processing CreateTablet for tablet 879666f362ba4a2db3fdfe6738b37dec (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260807 01:46:23.225348 27206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 879666f362ba4a2db3fdfe6738b37dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.211292 27205 tablet_service.cc:1511] Processing CreateTablet for tablet bf07cc968f3343869d12c10e81ce75f9 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260807 01:46:23.227632 27404 tablet_bootstrap.cc:654] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:23.207962 27201 tablet_service.cc:1511] Processing CreateTablet for tablet 6b0f72aee1954e3ab89690c7cfb93677 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260807 01:46:23.203408 27324 rpcz_store.cc:276] 0807 01:46:22.074655 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:22.655968 (+581313us) service_pool.cc:224] Handling call
0807 01:46:22.743089 (+ 87121us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:22.743200 (+ 111us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:22.952036 (+208836us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.176907 (+224871us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.202649 (+ 25742us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:46:23.234477 27094 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41078 (request call id 27) took 1001 ms. Trace:
I20260807 01:46:23.236971 27092 tablet_service.cc:1511] Processing CreateTablet for tablet 6420178dae2448f78216552b1b03c6e6 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260807 01:46:23.238642 27092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6420178dae2448f78216552b1b03c6e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.237628 27094 rpcz_store.cc:276] 0807 01:46:22.233030 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:22.760176 (+527146us) service_pool.cc:224] Handling call
0807 01:46:22.833930 (+ 73754us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:22.834053 (+ 123us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:22.862832 (+ 28779us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.233763 (+370931us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.234385 (+ 622us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9413}
I20260807 01:46:23.242324 27409 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:23.242756 27324 tablet_service.cc:1511] Processing CreateTablet for tablet bf9ccfe7f7e64481b739c6efef03b95a (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260807 01:46:23.242849 27409 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.244246 27324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf9ccfe7f7e64481b739c6efef03b95a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.244786 27205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf07cc968f3343869d12c10e81ce75f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.245098 27409 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:23.246031 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:23.247455 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:23.248443 27094 tablet_service.cc:1511] Processing CreateTablet for tablet 54eb75662df043488517f09bff1dfcff (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260807 01:46:23.249830 27094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54eb75662df043488517f09bff1dfcff. 1 dirs total, 0 dirs full, 0 dirs failed
W20260807 01:46:23.250955 27036 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 01:46:23.252097 27309 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:59474 (request call id 36) took 1174 ms. Trace:
W20260807 01:46:23.252132 27170 leader_election.cc:343] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:23.252835 27201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b0f72aee1954e3ab89690c7cfb93677. 1 dirs total, 0 dirs full, 0 dirs failed
W20260807 01:46:23.253193 27172 leader_election.cc:343] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:23.253631 27172 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:23.254271 27210 tablet_service.cc:1511] Processing CreateTablet for tablet fabde7ae28974f7380cb7641ab9fd6b1 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260807 01:46:23.255148 27409 raft_consensus.cc:2749] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:23.255699 27210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fabde7ae28974f7380cb7641ab9fd6b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.259613 27203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53780 (request call id 33) took 1021 ms. Trace:
I20260807 01:46:23.260047 27203 rpcz_store.cc:276] 0807 01:46:22.237985 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:22.739097 (+501112us) service_pool.cc:224] Handling call
0807 01:46:22.741213 (+ 2116us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:22.741307 (+ 94us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.059063 (+317756us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.259087 (+200024us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.259544 (+ 457us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:46:23.263134 27319 tablet_service.cc:1511] Processing CreateTablet for tablet f27eab393b1f4f21aad32707749effa3 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260807 01:46:23.273491 27319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f27eab393b1f4f21aad32707749effa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.272162 27407 tablet_bootstrap.cc:654] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:23.273885 27324 tablet_service.cc:1511] Processing CreateTablet for tablet 9febf660f0fd492dabb2b64043136fbe (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260807 01:46:23.203409 27313 rpcz_store.cc:276] 0807 01:46:22.105493 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:22.796499 (+691006us) service_pool.cc:224] Handling call
0807 01:46:22.816696 (+ 20197us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:22.816805 (+ 109us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:22.977434 (+160629us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.202533 (+225099us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.202984 (+ 451us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:46:23.270617 27089 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41078 (request call id 31) took 1017 ms. Trace:
I20260807 01:46:23.288502 27313 tablet_service.cc:1511] Processing CreateTablet for tablet 00395257beb442e892a4ec18c4c30d95 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260807 01:46:23.288941 27089 rpcz_store.cc:276] 0807 01:46:22.253397 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:22.816350 (+562953us) service_pool.cc:224] Handling call
0807 01:46:22.833428 (+ 17078us) spinlock_profiling.cc:243] Waited 4.34 ms on lock 0x7b5800024c98. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e084c 00007f7d522e040b 00007f7d5b24c219 00007f7d5a39f640 00007f7d5a484641 00007f7d5a4618d3 00007f7d5a410210 00007f7d53863bf4 00007f7d53863b82 00007f7d53863ab1
0807 01:46:22.854326 (+ 20898us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:22.854444 (+ 118us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.164149 (+309705us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.228751 (+ 64602us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.270535 (+ 41784us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9982208}
I20260807 01:46:23.292129 27310 tablet_service.cc:1511] Processing CreateTablet for tablet fabde7ae28974f7380cb7641ab9fd6b1 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260807 01:46:23.292678 27089 tablet_service.cc:1511] Processing CreateTablet for tablet b18526f467154c49b830de9b1b03066f (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260807 01:46:23.226593 27338 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:23.300124 27089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b18526f467154c49b830de9b1b03066f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.262099 27203 tablet_service.cc:1511] Processing CreateTablet for tablet 40f160d4dff84d8d9fb21b3489d01ff7 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260807 01:46:23.320418 27203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40f160d4dff84d8d9fb21b3489d01ff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.320801 27405 tablet_bootstrap.cc:654] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:23.294123 27310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fabde7ae28974f7380cb7641ab9fd6b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.296916 27211 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53780 (request call id 35) took 1041 ms. Trace:
I20260807 01:46:23.322857 27211 rpcz_store.cc:276] 0807 01:46:22.255708 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:22.806376 (+550668us) service_pool.cc:224] Handling call
0807 01:46:22.821627 (+ 15251us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:22.821719 (+ 92us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:22.947951 (+126232us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.296032 (+348081us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.296843 (+ 811us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:46:23.326553 27324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9febf660f0fd492dabb2b64043136fbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.252504 27309 rpcz_store.cc:276] 0807 01:46:22.077853 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:22.762631 (+684778us) service_pool.cc:224] Handling call
0807 01:46:22.804057 (+ 41426us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b5800003698. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e0521 00007f7d522e0b52 00007f7d5b275159 00007f7d5a39f4e8 00007f7d5a461990 00007f7d5a410210 00007f7d53863bf4 00007f7d53863b82 00007f7d53863ab1 00007f7d53863a2c
0807 01:46:22.804128 (+ 71us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:22.804240 (+ 112us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.156772 (+352532us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.251561 (+ 94789us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.252033 (+ 472us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59465984}
I20260807 01:46:23.302462 27098 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41078 (request call id 37) took 1035 ms. Trace:
I20260807 01:46:23.334723 27098 rpcz_store.cc:276] 0807 01:46:22.267392 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:23.073318 (+805926us) service_pool.cc:224] Handling call
0807 01:46:23.094885 (+ 21567us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:23.094986 (+ 101us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.284178 (+189192us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.301671 (+ 17493us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.302358 (+ 687us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:46:23.325659 27211 tablet_service.cc:1511] Processing CreateTablet for tablet bf9ccfe7f7e64481b739c6efef03b95a (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260807 01:46:23.339196 27098 tablet_service.cc:1511] Processing CreateTablet for tablet 6b0f72aee1954e3ab89690c7cfb93677 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260807 01:46:23.341269 27211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf9ccfe7f7e64481b739c6efef03b95a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.341624 27098 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b0f72aee1954e3ab89690c7cfb93677. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.332477 26952 catalog_manager.cc:5697] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:23.349615 27204 tablet_service.cc:1511] Processing CreateTablet for tablet f27eab393b1f4f21aad32707749effa3 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260807 01:46:23.357191 27309 tablet_service.cc:1511] Processing CreateTablet for tablet 4098be3cca39416abf4b724092db65e4 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260807 01:46:23.371317 27313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00395257beb442e892a4ec18c4c30d95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.395023 27309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4098be3cca39416abf4b724092db65e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.389704 27091 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41078 (request call id 35) took 1121 ms. Trace:
I20260807 01:46:23.479842 27091 rpcz_store.cc:276] 0807 01:46:22.266766 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:23.071370 (+804604us) service_pool.cc:224] Handling call
0807 01:46:23.113506 (+ 42136us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:23.113609 (+ 103us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.153564 (+ 39955us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.359395 (+205831us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.388499 (+ 29104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:46:23.393062 27094 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41078 (request call id 43) took 1037 ms. Trace:
I20260807 01:46:23.482549 27094 rpcz_store.cc:276] 0807 01:46:22.354669 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:23.247339 (+892670us) service_pool.cc:224] Handling call
0807 01:46:23.249323 (+ 1984us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:23.249426 (+ 103us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.277946 (+ 28520us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.373589 (+ 95643us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.392158 (+ 18569us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:46:23.395241 27197 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53780 (request call id 41) took 1078 ms. Trace:
I20260807 01:46:23.485256 27201 tablet_service.cc:1511] Processing CreateTablet for tablet 9febf660f0fd492dabb2b64043136fbe (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260807 01:46:23.488080 27197 rpcz_store.cc:276] 0807 01:46:22.316955 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:22.909926 (+592971us) service_pool.cc:224] Handling call
0807 01:46:22.985956 (+ 76030us) spinlock_profiling.cc:243] Waited 6.81 ms on lock 0x7b5800135398. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e084c 00007f7d522e040b 00007f7d5b24c219 00007f7d5a39f640 00007f7d5a484641 00007f7d5a4618d3 00007f7d5a410210 00007f7d53863bf4 00007f7d53863b82 00007f7d53863ab1
0807 01:46:23.016109 (+ 30153us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:23.016237 (+ 128us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.282424 (+266187us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.384545 (+102121us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.395123 (+ 10578us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15670528}
I20260807 01:46:23.394315 27204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f27eab393b1f4f21aad32707749effa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.491879 27087 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41078 (request call id 38) took 1224 ms. Trace:
I20260807 01:46:23.492545 27087 rpcz_store.cc:276] 0807 01:46:22.267666 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:23.079913 (+812247us) service_pool.cc:224] Handling call
0807 01:46:23.082044 (+ 2131us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:23.082146 (+ 102us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.262711 (+180565us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.490231 (+227520us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.491607 (+ 1376us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:46:23.494776 27201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9febf660f0fd492dabb2b64043136fbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.494518 27197 tablet_service.cc:1511] Processing CreateTablet for tablet 00395257beb442e892a4ec18c4c30d95 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260807 01:46:23.437687 27312 tablet_service.cc:1511] Processing CreateTablet for tablet 2c3fdef784cc4afbb8dda87287a7eac0 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260807 01:46:23.433876 27324 tablet_service.cc:1511] Processing CreateTablet for tablet 9f9f47b3d3124bda8fe235e759b1d061 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260807 01:46:23.424566 27317 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:59474 (request call id 43) took 1040 ms. Trace:
I20260807 01:46:23.393491 27404 tablet_bootstrap.cc:492] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:23.506001 27317 rpcz_store.cc:276] 0807 01:46:22.383719 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:23.047680 (+663961us) service_pool.cc:224] Handling call
0807 01:46:23.060991 (+ 13311us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:23.061093 (+ 102us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.273251 (+212158us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.411157 (+137906us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.423969 (+ 12812us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:46:23.508312 27404 ts_tablet_manager.cc:1403] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.466s user 0.018s sys 0.004s
I20260807 01:46:23.426213 27320 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:59474 (request call id 37) took 1347 ms. Trace:
I20260807 01:46:23.509370 27317 tablet_service.cc:1511] Processing CreateTablet for tablet 89dc6a602e9e42399d533f67d4bd9bef (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260807 01:46:23.510213 27320 rpcz_store.cc:276] 0807 01:46:22.078616 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:22.793920 (+715304us) service_pool.cc:224] Handling call
0807 01:46:22.840709 (+ 46789us) spinlock_profiling.cc:243] Waited 1.8 ms on lock 0x7b5800003698. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e0521 00007f7d522e0b52 00007f7d5b275159 00007f7d5a39f4e8 00007f7d5a461990 00007f7d5a410210 00007f7d53863bf4 00007f7d53863b82 00007f7d53863ab1 00007f7d53863a2c
0807 01:46:22.840770 (+ 61us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:22.840858 (+ 88us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.224273 (+383415us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.425587 (+201314us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.426115 (+ 528us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4136320}
I20260807 01:46:23.405296 27407 tablet_bootstrap.cc:492] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:23.513646 27314 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:59474 (request call id 45) took 1061 ms. Trace:
I20260807 01:46:23.513581 27407 ts_tablet_manager.cc:1403] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.403s user 0.021s sys 0.017s
I20260807 01:46:23.515461 27320 tablet_service.cc:1511] Processing CreateTablet for tablet 2cfefe1ff8f14fb68678ac128e129d32 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260807 01:46:23.505366 27312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c3fdef784cc4afbb8dda87287a7eac0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.517150 27320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cfefe1ff8f14fb68678ac128e129d32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.515612 27091 tablet_service.cc:1511] Processing CreateTablet for tablet 879666f362ba4a2db3fdfe6738b37dec (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260807 01:46:23.522349 27324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f9f47b3d3124bda8fe235e759b1d061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.517467 27414 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:23.526376 27414 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.536581 27209 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53780 (request call id 36) took 1278 ms. Trace:
I20260807 01:46:23.521518 27197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00395257beb442e892a4ec18c4c30d95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.519534 27094 tablet_service.cc:1511] Processing CreateTablet for tablet bf07cc968f3343869d12c10e81ce75f9 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
W20260807 01:46:23.553416 26931 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 50289.9us
I20260807 01:46:23.554306 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:23.557224 27287 leader_election.cc:343] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:23.518656 27314 rpcz_store.cc:276] 0807 01:46:22.452362 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:23.058507 (+606145us) service_pool.cc:224] Handling call
0807 01:46:23.095862 (+ 37355us) spinlock_profiling.cc:243] Waited 2.1 ms on lock 0x7b5800003698. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e0521 00007f7d522e03d2 00007f7d5b24c219 00007f7d5a39f640 00007f7d5a484641 00007f7d5a4618d3 00007f7d5a410210 00007f7d53863bf4 00007f7d53863b82 00007f7d53863ab1
0807 01:46:23.137479 (+ 41617us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:23.177083 (+ 39604us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.250063 (+ 72980us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.512448 (+262385us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.513397 (+ 949us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4820352}
I20260807 01:46:23.561753 27091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 879666f362ba4a2db3fdfe6738b37dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.561419 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:23.561290 27087 tablet_service.cc:1511] Processing CreateTablet for tablet 40f160d4dff84d8d9fb21b3489d01ff7 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260807 01:46:23.567325 27317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89dc6a602e9e42399d533f67d4bd9bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.572299 27086 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41078 (request call id 32) took 1318 ms. Trace:
I20260807 01:46:23.573230 27094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf07cc968f3343869d12c10e81ce75f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.580614 27088 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41078 (request call id 36) took 1313 ms. Trace:
I20260807 01:46:23.581146 27088 rpcz_store.cc:276] 0807 01:46:22.267109 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:23.055619 (+788510us) service_pool.cc:224] Handling call
0807 01:46:23.057543 (+ 1924us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:23.057644 (+ 101us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.303557 (+245913us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.579814 (+276257us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.580415 (+ 601us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:46:23.550371 27414 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:46:23.572144 27287 leader_election.cc:343] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:23.577598 27086 rpcz_store.cc:276] 0807 01:46:22.253687 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:22.841798 (+588111us) service_pool.cc:224] Handling call
0807 01:46:22.845543 (+ 3745us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:22.845659 (+ 116us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.224975 (+379316us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.534343 (+309368us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.572084 (+ 37741us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1628416}
I20260807 01:46:23.582412 27287 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:23.583500 27405 tablet_bootstrap.cc:492] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:23.584065 27414 raft_consensus.cc:2749] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:23.543304 27209 rpcz_store.cc:276] 0807 01:46:22.257609 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:22.766092 (+508483us) service_pool.cc:224] Handling call
0807 01:46:22.831318 (+ 65226us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:22.855003 (+ 23685us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.186345 (+331342us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.534984 (+348639us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.536234 (+ 1250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:46:23.584393 27086 tablet_service.cc:1511] Processing CreateTablet for tablet fabde7ae28974f7380cb7641ab9fd6b1 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260807 01:46:23.599323 27209 tablet_service.cc:1511] Processing CreateTablet for tablet 4098be3cca39416abf4b724092db65e4 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260807 01:46:23.585529 27404 raft_consensus.cc:359] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.601503 27404 raft_consensus.cc:385] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:23.595605 27199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53780 (request call id 44) took 1189 ms. Trace:
I20260807 01:46:23.601145 27209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4098be3cca39416abf4b724092db65e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.573757 27407 raft_consensus.cc:359] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.581753 27090 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41078 (request call id 39) took 1313 ms. Trace:
I20260807 01:46:23.584316 27405 ts_tablet_manager.cc:1403] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.412s user 0.014s sys 0.009s
I20260807 01:46:23.610589 27404 raft_consensus.cc:740] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:23.603166 27205 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53780 (request call id 46) took 1119 ms. Trace:
I20260807 01:46:23.589200 27087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40f160d4dff84d8d9fb21b3489d01ff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.617880 27205 rpcz_store.cc:276] 0807 01:46:22.483696 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:23.210313 (+726617us) service_pool.cc:224] Handling call
0807 01:46:23.230266 (+ 19953us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:23.230393 (+ 127us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.555481 (+325088us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.602472 (+ 46991us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.602985 (+ 513us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:46:23.619589 27404 consensus_queue.cc:260] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.628319 27205 tablet_service.cc:1511] Processing CreateTablet for tablet 9f9f47b3d3124bda8fe235e759b1d061 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260807 01:46:23.615020 27405 raft_consensus.cc:359] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.611600 27407 raft_consensus.cc:385] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:23.626019 27086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fabde7ae28974f7380cb7641ab9fd6b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.631024 27407 raft_consensus.cc:740] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:23.632217 27407 consensus_queue.cc:260] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.630545 27405 raft_consensus.cc:385] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:23.611397 27199 rpcz_store.cc:276] 0807 01:46:22.406466 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:23.111664 (+705198us) service_pool.cc:224] Handling call
0807 01:46:23.113498 (+ 1834us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:23.113600 (+ 102us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.366010 (+252410us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.594722 (+228712us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.595388 (+ 666us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:46:23.639292 27199 tablet_service.cc:1511] Processing CreateTablet for tablet 89dc6a602e9e42399d533f67d4bd9bef (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260807 01:46:23.611788 27090 rpcz_store.cc:276] 0807 01:46:22.267954 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:23.084556 (+816602us) service_pool.cc:224] Handling call
0807 01:46:23.115982 (+ 31426us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:23.120081 (+ 4099us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.284155 (+164074us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.580964 (+296809us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.581561 (+ 597us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:46:23.644081 27090 tablet_service.cc:1511] Processing CreateTablet for tablet f27eab393b1f4f21aad32707749effa3 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
W20260807 01:46:23.645643 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 73538.8us
I20260807 01:46:23.646660 27205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f9f47b3d3124bda8fe235e759b1d061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.585716 27088 tablet_service.cc:1511] Processing CreateTablet for tablet bf9ccfe7f7e64481b739c6efef03b95a (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260807 01:46:23.634402 27405 raft_consensus.cc:740] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:23.650609 27199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89dc6a602e9e42399d533f67d4bd9bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.645963 27090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f27eab393b1f4f21aad32707749effa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.635882 27407 ts_tablet_manager.cc:1434] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.121s user 0.002s sys 0.007s
I20260807 01:46:23.656002 27407 tablet_bootstrap.cc:492] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:23.649076 27404 ts_tablet_manager.cc:1434] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.140s user 0.010s sys 0.003s
I20260807 01:46:23.668113 27404 tablet_bootstrap.cc:492] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:23.649071 27405 consensus_queue.cc:260] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.670971 27089 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41078 (request call id 44) took 1273 ms. Trace:
I20260807 01:46:23.671413 27089 rpcz_store.cc:276] 0807 01:46:22.397887 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:23.291434 (+893547us) service_pool.cc:224] Handling call
0807 01:46:23.299226 (+ 7792us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:23.299416 (+ 190us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.576209 (+276793us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.664219 (+ 88010us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.670766 (+ 6547us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":169}
I20260807 01:46:23.672652 27405 ts_tablet_manager.cc:1434] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.061s user 0.008s sys 0.000s
I20260807 01:46:23.673647 27088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf9ccfe7f7e64481b739c6efef03b95a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.632241 27210 tablet_service.cc:1511] Processing CreateTablet for tablet 2c3fdef784cc4afbb8dda87287a7eac0 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260807 01:46:23.674844 27089 tablet_service.cc:1511] Processing CreateTablet for tablet 9febf660f0fd492dabb2b64043136fbe (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260807 01:46:23.675510 27210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c3fdef784cc4afbb8dda87287a7eac0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.676571 27093 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41078 (request call id 41) took 1360 ms. Trace:
I20260807 01:46:23.678058 27093 rpcz_store.cc:276] 0807 01:46:22.316536 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:23.205264 (+888728us) service_pool.cc:224] Handling call
0807 01:46:23.207502 (+ 2238us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:23.207634 (+ 132us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.540002 (+332368us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.664941 (+124939us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.676389 (+ 11448us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5678}
I20260807 01:46:23.681447 27093 tablet_service.cc:1511] Processing CreateTablet for tablet 00395257beb442e892a4ec18c4c30d95 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260807 01:46:23.682359 27089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9febf660f0fd492dabb2b64043136fbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.684760 27092 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41078 (request call id 42) took 1337 ms. Trace:
I20260807 01:46:23.684952 27094 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41078 (request call id 46) took 1205 ms. Trace:
I20260807 01:46:23.685225 27092 rpcz_store.cc:276] 0807 01:46:22.347234 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:23.235959 (+888725us) service_pool.cc:224] Handling call
0807 01:46:23.238112 (+ 2153us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:23.238248 (+ 136us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.576226 (+337978us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.684079 (+107853us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.684560 (+ 481us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:46:23.686117 27093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00395257beb442e892a4ec18c4c30d95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.688403 27094 rpcz_store.cc:276] 0807 01:46:22.479123 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:23.484583 (+1005460us) service_pool.cc:224] Handling call
0807 01:46:23.572538 (+ 87955us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b5800024c98. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e0c68 00007f7d522e0b8b 00007f7d5b275159 00007f7d5a39f4e8 00007f7d5a461990 00007f7d5a410210 00007f7d53863bf4 00007f7d53863b82 00007f7d53863ab1 00007f7d53863a2c
0807 01:46:23.572617 (+ 79us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:23.572759 (+ 142us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.621381 (+ 48622us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.661003 (+ 39622us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.684868 (+ 23865us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3251072}
I20260807 01:46:23.692356 27409 consensus_queue.cc:1048] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:23.698980 27091 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41078 (request call id 47) took 1186 ms. Trace:
I20260807 01:46:23.699564 27091 rpcz_store.cc:276] 0807 01:46:22.512699 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:23.514439 (+1001740us) service_pool.cc:224] Handling call
0807 01:46:23.531716 (+ 17277us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:23.561152 (+ 29436us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.681101 (+119949us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.698400 (+ 17299us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.698908 (+ 508us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:46:23.702682 27203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53780 (request call id 48) took 1027 ms. Trace:
I20260807 01:46:23.703127 27203 rpcz_store.cc:276] 0807 01:46:22.675403 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:23.261131 (+585728us) service_pool.cc:224] Handling call
0807 01:46:23.319799 (+ 58668us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:23.319948 (+ 149us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.538926 (+218978us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.648490 (+109564us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.702516 (+ 54026us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52986}
I20260807 01:46:23.704073 27211 tablet_service.cc:1511] Processing CreateTablet for tablet 2cfefe1ff8f14fb68678ac128e129d32 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260807 01:46:23.705230 27405 tablet_bootstrap.cc:492] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:23.705672 27211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cfefe1ff8f14fb68678ac128e129d32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.704335 27407 tablet_bootstrap.cc:654] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:23.727097 27098 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41078 (request call id 45) took 1302 ms. Trace:
I20260807 01:46:23.727556 27098 rpcz_store.cc:276] 0807 01:46:22.425059 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:23.337063 (+912004us) service_pool.cc:224] Handling call
0807 01:46:23.340768 (+ 3705us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:23.340903 (+ 135us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.611896 (+270993us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.726071 (+114175us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.726892 (+ 821us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:46:23.737884 27206 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53780 (request call id 47) took 1222 ms. Trace:
I20260807 01:46:23.738372 27206 rpcz_store.cc:276] 0807 01:46:22.514942 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:23.222728 (+707786us) service_pool.cc:224] Handling call
0807 01:46:23.224842 (+ 2114us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:23.224943 (+ 101us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.596218 (+371275us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.701554 (+105336us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.737688 (+ 36134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35442}
I20260807 01:46:23.742931 27092 tablet_service.cc:1511] Processing CreateTablet for tablet 4098be3cca39416abf4b724092db65e4 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260807 01:46:23.744072 27092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4098be3cca39416abf4b724092db65e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.748777 27405 tablet_bootstrap.cc:654] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:23.752008 27098 tablet_service.cc:1511] Processing CreateTablet for tablet 9f9f47b3d3124bda8fe235e759b1d061 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260807 01:46:23.755383 27098 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f9f47b3d3124bda8fe235e759b1d061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.761106 27094 tablet_service.cc:1511] Processing CreateTablet for tablet 2c3fdef784cc4afbb8dda87287a7eac0 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260807 01:46:23.775002 27091 tablet_service.cc:1511] Processing CreateTablet for tablet 89dc6a602e9e42399d533f67d4bd9bef (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260807 01:46:23.780305 27094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c3fdef784cc4afbb8dda87287a7eac0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.787470 27088 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41078 (request call id 50) took 1014 ms. Trace:
I20260807 01:46:23.789865 27088 rpcz_store.cc:276] 0807 01:46:22.772853 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:23.584689 (+811836us) service_pool.cc:224] Handling call
0807 01:46:23.647435 (+ 62746us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:23.647564 (+ 129us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.756600 (+109036us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.786801 (+ 30201us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.787392 (+ 591us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:46:23.797400 27088 tablet_service.cc:1511] Processing CreateTablet for tablet 2cfefe1ff8f14fb68678ac128e129d32 (DEFAULT_TABLE table=test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260807 01:46:23.798955 27446 raft_consensus.cc:493] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:23.801645 27446 raft_consensus.cc:515] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.804277 27446 leader_election.cc:290] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:23.807271 27087 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41078 (request call id 48) took 1190 ms. Trace:
I20260807 01:46:23.807587 27087 rpcz_store.cc:276] 0807 01:46:22.616527 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:23.560038 (+943511us) service_pool.cc:224] Handling call
0807 01:46:23.588179 (+ 28141us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:23.588364 (+ 185us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.739140 (+150776us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.806879 (+ 67739us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.807218 (+ 339us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:46:23.808535 27404 tablet_bootstrap.cc:654] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:23.818238 27091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89dc6a602e9e42399d533f67d4bd9bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.816089 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:23.827756 27230 raft_consensus.cc:2468] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:23.838105 27287 leader_election.cc:304] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:23.845898 27446 raft_consensus.cc:2804] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:23.846576 27446 raft_consensus.cc:493] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:23.846933 27446 raft_consensus.cc:3060] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:23.864511 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:23.865300 27109 raft_consensus.cc:2468] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:23.865455 27407 tablet_bootstrap.cc:492] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:23.867450 27407 ts_tablet_manager.cc:1403] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.212s user 0.017s sys 0.029s
I20260807 01:46:23.874343 27086 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:41078 (request call id 49) took 1219 ms. Trace:
I20260807 01:46:23.875293 27446 raft_consensus.cc:515] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.856678 27409 consensus_queue.cc:1048] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260807 01:46:23.875910 27407 raft_consensus.cc:359] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.876682 27407 raft_consensus.cc:385] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:23.877003 27407 raft_consensus.cc:740] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:23.857785 27088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cfefe1ff8f14fb68678ac128e129d32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:23.874784 27086 rpcz_store.cc:276] 0807 01:46:22.654629 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:23.583056 (+928427us) service_pool.cc:224] Handling call
0807 01:46:23.599397 (+ 16341us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:23.599810 (+ 413us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.683864 (+ 84054us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.873647 (+189783us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.874262 (+ 615us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:46:23.883301 27407 consensus_queue.cc:260] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.886919 27407 ts_tablet_manager.cc:1434] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.019s user 0.006s sys 0.001s
I20260807 01:46:23.888126 27407 tablet_bootstrap.cc:492] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:23.888866 27405 tablet_bootstrap.cc:492] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:23.889410 27405 ts_tablet_manager.cc:1403] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.184s user 0.018s sys 0.013s
I20260807 01:46:23.892836 27405 raft_consensus.cc:359] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.893438 27420 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:23.893640 27405 raft_consensus.cc:385] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:23.893949 27420 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.896437 27420 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:23.897393 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:23.898219 27230 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
I20260807 01:46:23.898228 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:23.900899 27446 leader_election.cc:290] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:46:23.902683 27063 leader_election.cc:343] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:23.902976 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:23.903599 27230 raft_consensus.cc:3060] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:23.903838 27064 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:46:23.905385 27420 raft_consensus.cc:2804] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:23.905784 27420 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:23.906126 27420 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:23.912746 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:23.893957 27405 raft_consensus.cc:740] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:23.913429 27110 raft_consensus.cc:3060] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:23.913802 27405 consensus_queue.cc:260] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.920173 27405 ts_tablet_manager.cc:1434] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.030s user 0.008s sys 0.000s
I20260807 01:46:23.921720 27405 tablet_bootstrap.cc:492] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:23.921795 27110 raft_consensus.cc:2468] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:23.923476 27287 leader_election.cc:304] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:23.924193 27404 tablet_bootstrap.cc:492] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:23.924474 27446 raft_consensus.cc:2804] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:23.924484 27407 tablet_bootstrap.cc:654] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:23.925072 27446 raft_consensus.cc:697] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:23.926018 27446 consensus_queue.cc:237] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.927289 27420 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.929893 27230 raft_consensus.cc:2468] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:23.924718 27404 ts_tablet_manager.cc:1403] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.257s user 0.026s sys 0.020s
I20260807 01:46:23.939486 27407 tablet_bootstrap.cc:492] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:23.939949 27407 ts_tablet_manager.cc:1403] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.052s user 0.014s sys 0.001s
I20260807 01:46:23.942938 27407 raft_consensus.cc:359] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.943529 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:23.943738 27407 raft_consensus.cc:385] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:23.943467 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:23.944360 27229 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
W20260807 01:46:23.945181 27063 leader_election.cc:343] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:23.945883 27420 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:23.947837 26954 catalog_manager.cc:5697] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:23.950290 27407 raft_consensus.cc:740] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:23.951527 27407 consensus_queue.cc:260] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.937435 27404 raft_consensus.cc:359] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.953612 27404 raft_consensus.cc:385] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:23.953953 27404 raft_consensus.cc:740] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:23.954185 27407 ts_tablet_manager.cc:1434] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.014s user 0.005s sys 0.004s
I20260807 01:46:23.955799 27407 tablet_bootstrap.cc:492] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:23.956245 27404 consensus_queue.cc:260] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.950529 27229 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:23.958942 27064 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:46:23.959941 27405 tablet_bootstrap.cc:654] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:23.960350 27420 raft_consensus.cc:2804] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:23.960922 27420 raft_consensus.cc:697] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:23.961839 27420 consensus_queue.cc:237] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.962859 27407 tablet_bootstrap.cc:654] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:23.964380 27404 ts_tablet_manager.cc:1434] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20260807 01:46:23.965552 27404 tablet_bootstrap.cc:492] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:23.986124 27404 tablet_bootstrap.cc:654] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:23.989739 27405 tablet_bootstrap.cc:492] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:23.991780 27405 ts_tablet_manager.cc:1403] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.070s user 0.017s sys 0.000s
I20260807 01:46:23.992074 27407 tablet_bootstrap.cc:492] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:23.992616 27407 ts_tablet_manager.cc:1403] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.037s user 0.030s sys 0.003s
I20260807 01:46:23.995713 27405 raft_consensus.cc:359] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:23.995841 27407 raft_consensus.cc:359] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } }
I20260807 01:46:23.996618 27405 raft_consensus.cc:385] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:23.996709 27407 raft_consensus.cc:385] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:23.997042 27405 raft_consensus.cc:740] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:23.997120 27407 raft_consensus.cc:740] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:23.998057 27407 consensus_queue.cc:260] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } }
I20260807 01:46:23.998224 27405 consensus_queue.cc:260] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.000785 27407 ts_tablet_manager.cc:1434] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260807 01:46:24.002039 27407 tablet_bootstrap.cc:492] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:24.015571 27405 ts_tablet_manager.cc:1434] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.023s user 0.010s sys 0.000s
I20260807 01:46:24.017549 26952 catalog_manager.cc:5697] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:24.022420 27405 tablet_bootstrap.cc:492] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:24.022688 27407 tablet_bootstrap.cc:654] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:24.030709 27404 tablet_bootstrap.cc:492] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:24.031622 27404 ts_tablet_manager.cc:1403] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.066s user 0.017s sys 0.019s
I20260807 01:46:24.038599 27404 raft_consensus.cc:359] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.039867 27404 raft_consensus.cc:385] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:24.040414 27405 tablet_bootstrap.cc:654] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:24.040344 27404 raft_consensus.cc:740] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:24.042058 27404 consensus_queue.cc:260] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.047451 27404 ts_tablet_manager.cc:1434] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.015s user 0.010s sys 0.006s
I20260807 01:46:24.050489 27404 tablet_bootstrap.cc:492] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:24.074472 27404 tablet_bootstrap.cc:654] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:24.098544 27404 tablet_bootstrap.cc:492] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:24.099175 27405 tablet_bootstrap.cc:492] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:24.099334 27404 ts_tablet_manager.cc:1403] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.049s user 0.018s sys 0.026s
I20260807 01:46:24.099982 27405 ts_tablet_manager.cc:1403] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.078s user 0.043s sys 0.032s
I20260807 01:46:24.102528 27407 tablet_bootstrap.cc:492] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:24.103379 27407 ts_tablet_manager.cc:1403] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.102s user 0.026s sys 0.054s
I20260807 01:46:24.105306 27404 raft_consensus.cc:359] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } }
I20260807 01:46:24.106338 27404 raft_consensus.cc:385] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:24.106900 27404 raft_consensus.cc:740] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:24.108055 27404 consensus_queue.cc:260] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } }
I20260807 01:46:24.107553 27405 raft_consensus.cc:359] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.109184 27405 raft_consensus.cc:385] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:24.109697 27405 raft_consensus.cc:740] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:24.111446 27446 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:24.111635 27404 ts_tablet_manager.cc:1434] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.012s user 0.006s sys 0.006s
I20260807 01:46:24.110986 27407 raft_consensus.cc:359] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.112294 27407 raft_consensus.cc:385] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:24.112051 27446 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.111011 27405 consensus_queue.cc:260] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.115511 27446 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:24.112785 27407 raft_consensus.cc:740] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:24.117377 27407 consensus_queue.cc:260] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.118306 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:24.118876 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:24.119328 27230 raft_consensus.cc:2468] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:24.119771 27109 raft_consensus.cc:2468] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:24.121636 27407 ts_tablet_manager.cc:1434] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.018s user 0.012s sys 0.001s
I20260807 01:46:24.122917 27287 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:24.125283 27405 ts_tablet_manager.cc:1434] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.025s user 0.005s sys 0.008s
I20260807 01:46:24.125211 27446 raft_consensus.cc:2804] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:24.113873 27404 tablet_bootstrap.cc:492] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:24.126420 27446 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:24.126973 27446 raft_consensus.cc:3060] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:24.131531 27407 tablet_bootstrap.cc:492] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:24.140578 27446 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.142601 27404 tablet_bootstrap.cc:654] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:24.146247 27407 tablet_bootstrap.cc:654] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:24.146216 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:24.147396 27109 raft_consensus.cc:3060] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:24.159063 27446 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:24.161247 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:24.162540 27230 raft_consensus.cc:3060] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:24.160575 27405 tablet_bootstrap.cc:492] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:24.167755 27446 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:24.168812 27446 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } }
I20260807 01:46:24.181666 27109 raft_consensus.cc:2468] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:24.185700 27446 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:46:24.186636 27404 tablet_bootstrap.cc:492] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:24.187117 27404 ts_tablet_manager.cc:1403] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.074s user 0.012s sys 0.041s
I20260807 01:46:24.187327 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:24.189275 27287 leader_election.cc:343] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:24.193508 27405 tablet_bootstrap.cc:654] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:24.194973 27230 raft_consensus.cc:2468] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:24.197283 27404 raft_consensus.cc:359] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.197753 27287 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:24.197973 27407 tablet_bootstrap.cc:492] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:24.197988 27404 raft_consensus.cc:385] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:24.198601 27404 raft_consensus.cc:740] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:24.198761 27407 ts_tablet_manager.cc:1403] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.067s user 0.024s sys 0.022s
I20260807 01:46:24.195425 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:24.199430 27404 consensus_queue.cc:260] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.200073 27110 raft_consensus.cc:2468] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:24.200769 27414 raft_consensus.cc:2804] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:24.202172 27407 raft_consensus.cc:359] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.202929 27407 raft_consensus.cc:385] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:24.203214 27407 raft_consensus.cc:740] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:24.204209 27407 consensus_queue.cc:260] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.206496 27407 ts_tablet_manager.cc:1434] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 01:46:24.207521 27407 tablet_bootstrap.cc:492] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:24.207765 27404 ts_tablet_manager.cc:1434] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.020s user 0.007s sys 0.000s
I20260807 01:46:24.210333 27404 tablet_bootstrap.cc:492] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:24.213727 27407 tablet_bootstrap.cc:654] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:24.202080 27287 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:24.216487 27446 raft_consensus.cc:2804] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:24.217031 27446 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:24.220743 27414 raft_consensus.cc:697] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:24.226585 27407 tablet_bootstrap.cc:492] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:24.227273 27407 ts_tablet_manager.cc:1403] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.020s user 0.014s sys 0.004s
I20260807 01:46:24.226888 27414 consensus_queue.cc:237] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.230966 27407 raft_consensus.cc:359] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.231987 27407 raft_consensus.cc:385] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:24.232339 27407 raft_consensus.cc:740] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:24.233219 27407 consensus_queue.cc:260] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.236127 27407 ts_tablet_manager.cc:1434] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260807 01:46:24.223047 27404 tablet_bootstrap.cc:654] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:24.225839 27446 raft_consensus.cc:3060] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:24.237602 27407 tablet_bootstrap.cc:492] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:24.245187 27407 tablet_bootstrap.cc:654] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:24.251628 27446 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } }
W20260807 01:46:24.254356 27036 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 01:46:24.255474 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
W20260807 01:46:24.258311 27287 leader_election.cc:343] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:24.262545 27405 tablet_bootstrap.cc:492] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:24.263214 27405 ts_tablet_manager.cc:1403] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.133s user 0.038s sys 0.044s
I20260807 01:46:24.265275 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:24.266062 27110 raft_consensus.cc:3060] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:24.266574 27405 raft_consensus.cc:359] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.267397 27405 raft_consensus.cc:385] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:24.267675 27405 raft_consensus.cc:740] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:24.268635 27405 consensus_queue.cc:260] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.274441 27446 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:46:24.275386 27110 raft_consensus.cc:2468] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:24.283272 27287 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:24.276011 27405 ts_tablet_manager.cc:1434] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260807 01:46:24.285498 27405 tablet_bootstrap.cc:492] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:24.286536 27461 raft_consensus.cc:493] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:24.287047 27461 raft_consensus.cc:515] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.289441 27461 leader_election.cc:290] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:24.291479 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:24.292243 27230 raft_consensus.cc:2468] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
I20260807 01:46:24.293013 27446 raft_consensus.cc:2804] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:24.293570 27446 raft_consensus.cc:697] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:24.294176 27064 leader_election.cc:304] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:24.294448 27446 consensus_queue.cc:237] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } }
I20260807 01:46:24.294883 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:24.295173 27461 raft_consensus.cc:2804] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:24.295545 27461 raft_consensus.cc:493] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:24.298430 27404 tablet_bootstrap.cc:492] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:24.298856 27409 raft_consensus.cc:493] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:24.295519 27338 raft_consensus.cc:2468] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
I20260807 01:46:24.299306 27409 raft_consensus.cc:515] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.296046 27461 raft_consensus.cc:3060] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:24.301451 27409 leader_election.cc:290] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:24.301988 27405 tablet_bootstrap.cc:654] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:24.303246 26954 catalog_manager.cc:5697] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:24.304148 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:24.304821 27338 raft_consensus.cc:2468] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:24.306610 27170 leader_election.cc:304] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:24.307590 27409 raft_consensus.cc:2804] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:24.307987 27409 raft_consensus.cc:493] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:24.308316 27409 raft_consensus.cc:3060] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:24.313855 27405 tablet_bootstrap.cc:492] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:24.314476 27405 ts_tablet_manager.cc:1403] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.029s user 0.007s sys 0.006s
I20260807 01:46:24.314589 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:24.315382 27110 raft_consensus.cc:2468] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:24.317322 27404 ts_tablet_manager.cc:1403] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.107s user 0.022s sys 0.012s
I20260807 01:46:24.318886 27405 raft_consensus.cc:359] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.319891 27405 raft_consensus.cc:385] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:24.320217 27405 raft_consensus.cc:740] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:24.320413 27404 raft_consensus.cc:359] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.321552 27407 tablet_bootstrap.cc:492] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:24.321102 27405 consensus_queue.cc:260] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.322202 27407 ts_tablet_manager.cc:1403] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.085s user 0.026s sys 0.019s
I20260807 01:46:24.323513 27409 raft_consensus.cc:515] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.310529 27461 raft_consensus.cc:515] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.321127 27404 raft_consensus.cc:385] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:24.326697 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:24.327314 27110 raft_consensus.cc:3060] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:24.327482 27405 ts_tablet_manager.cc:1434] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.013s user 0.007s sys 0.001s
I20260807 01:46:24.328162 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:24.328784 27405 tablet_bootstrap.cc:492] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:24.328754 27230 raft_consensus.cc:3060] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:24.326269 27404 raft_consensus.cc:740] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:24.337877 27404 consensus_queue.cc:260] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.340488 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:24.341195 27338 raft_consensus.cc:3060] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:24.341377 27405 tablet_bootstrap.cc:654] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:24.344199 27453 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:24.344758 27453 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.348802 27230 raft_consensus.cc:2468] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:24.349188 27110 raft_consensus.cc:2468] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:24.351667 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:24.352295 27064 leader_election.cc:304] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:24.352804 27338 raft_consensus.cc:2468] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:24.353011 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:24.352414 27109 raft_consensus.cc:2468] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:24.354207 27170 leader_election.cc:304] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:24.354852 27172 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
W20260807 01:46:24.355543 27170 leader_election.cc:343] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:24.354199 27453 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:24.353276 27420 raft_consensus.cc:2804] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:24.349586 27409 leader_election.cc:290] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:24.355543 27450 raft_consensus.cc:2804] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:24.356887 27450 raft_consensus.cc:697] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:24.351621 27404 ts_tablet_manager.cc:1434] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.034s user 0.006s sys 0.001s
I20260807 01:46:24.357823 27450 consensus_queue.cc:237] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.359586 27461 leader_election.cc:290] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:24.350308 27407 raft_consensus.cc:359] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.360749 27407 raft_consensus.cc:385] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:24.361064 27407 raft_consensus.cc:740] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:24.361456 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:24.362272 27338 raft_consensus.cc:3060] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:24.356356 27453 raft_consensus.cc:2804] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:24.349982 26954 catalog_manager.cc:5697] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:24.362980 27420 raft_consensus.cc:697] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:24.364017 27420 consensus_queue.cc:237] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.361882 27407 consensus_queue.cc:260] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.358585 27404 tablet_bootstrap.cc:492] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:24.367292 27407 ts_tablet_manager.cc:1434] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.044s user 0.006s sys 0.000s
I20260807 01:46:24.363009 27453 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:24.368103 27453 raft_consensus.cc:3060] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:24.368345 27407 tablet_bootstrap.cc:492] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:24.375177 27407 tablet_bootstrap.cc:654] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:24.385408 27446 consensus_queue.cc:1048] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:24.383173 27338 raft_consensus.cc:2468] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:24.391860 27404 tablet_bootstrap.cc:654] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:24.391970 26950 catalog_manager.cc:5697] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:24.394274 27407 tablet_bootstrap.cc:492] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:24.394858 27407 ts_tablet_manager.cc:1403] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.027s user 0.013s sys 0.005s
I20260807 01:46:24.395543 26952 catalog_manager.cc:5697] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:24.397933 27407 raft_consensus.cc:359] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.398754 27407 raft_consensus.cc:385] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:24.399039 27407 raft_consensus.cc:740] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:24.399845 27407 consensus_queue.cc:260] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.405079 27407 ts_tablet_manager.cc:1434] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20260807 01:46:24.405566 27405 tablet_bootstrap.cc:492] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:24.406124 27407 tablet_bootstrap.cc:492] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:24.406394 27453 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.407105 27404 tablet_bootstrap.cc:492] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:24.407642 27404 ts_tablet_manager.cc:1403] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.049s user 0.013s sys 0.005s
I20260807 01:46:24.406090 27405 ts_tablet_manager.cc:1403] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.078s user 0.016s sys 0.007s
I20260807 01:46:24.409735 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:24.409982 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:24.410427 27110 raft_consensus.cc:3060] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
W20260807 01:46:24.411901 27170 leader_election.cc:343] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:24.412411 27405 raft_consensus.cc:359] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } }
I20260807 01:46:24.413131 27405 raft_consensus.cc:385] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:24.413429 27405 raft_consensus.cc:740] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:24.414136 27453 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:24.414942 27405 consensus_queue.cc:260] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } }
I20260807 01:46:24.416456 27404 raft_consensus.cc:359] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.417227 27404 raft_consensus.cc:385] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:24.417593 27404 raft_consensus.cc:740] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:24.418443 27404 consensus_queue.cc:260] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.420421 27404 ts_tablet_manager.cc:1434] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.012s user 0.003s sys 0.002s
I20260807 01:46:24.421576 27404 tablet_bootstrap.cc:492] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:24.431984 27494 raft_consensus.cc:493] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:24.432683 27494 raft_consensus.cc:515] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.417219 27405 ts_tablet_manager.cc:1434] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.008s user 0.002s sys 0.004s
I20260807 01:46:24.439061 27404 tablet_bootstrap.cc:654] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:24.444541 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:24.445633 27338 raft_consensus.cc:2468] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:24.439544 27405 tablet_bootstrap.cc:492] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:24.442986 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:24.447052 27109 raft_consensus.cc:2468] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:24.449231 27172 leader_election.cc:304] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:24.443375 27494 leader_election.cc:290] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:24.452194 27450 raft_consensus.cc:2804] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:24.452364 27420 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:24.452728 27450 raft_consensus.cc:493] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:24.453269 27450 raft_consensus.cc:3060] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:24.453251 27420 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.463567 27484 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:24.464274 27484 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.467124 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:24.467911 27420 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:46:24.470937 27064 leader_election.cc:343] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:24.472823 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:46:24.475020 27063 leader_election.cc:343] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:24.475812 27063 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:24.479156 27484 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:24.482050 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:24.483141 27230 raft_consensus.cc:2393] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 1: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:24.483556 27420 raft_consensus.cc:2749] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:24.487545 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:24.491439 27450 raft_consensus.cc:515] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.493686 27446 consensus_queue.cc:1048] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:24.499159 27287 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:24.500221 27405 tablet_bootstrap.cc:654] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:24.502673 27404 tablet_bootstrap.cc:492] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:24.503016 27407 tablet_bootstrap.cc:654] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:24.503528 27446 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:24.503218 27404 ts_tablet_manager.cc:1403] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.082s user 0.003s sys 0.020s
I20260807 01:46:24.510231 27495 consensus_queue.cc:1048] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260807 01:46:24.519057 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:24.520378 27108 raft_consensus.cc:3060] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:24.524820 27495 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:24.525589 27495 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.530066 27495 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:24.530296 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:24.531195 27229 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
I20260807 01:46:24.532792 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:24.533043 27064 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:24.534683 27495 raft_consensus.cc:2804] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:24.535357 27495 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:24.535899 27495 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:24.539429 27446 raft_consensus.cc:2749] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 01:46:24.543402 27063 leader_election.cc:343] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:24.546837 27404 raft_consensus.cc:359] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.547855 27404 raft_consensus.cc:385] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:24.548174 27404 raft_consensus.cc:740] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:24.548377 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:24.549297 27404 consensus_queue.cc:260] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.549649 27420 consensus_queue.cc:1048] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:24.542742 27450 leader_election.cc:290] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:24.515538 27110 raft_consensus.cc:2468] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:24.551698 27337 raft_consensus.cc:3060] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:24.558702 27404 ts_tablet_manager.cc:1434] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.052s user 0.010s sys 0.000s
I20260807 01:46:24.568452 27172 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:46:24.575342 27404 tablet_bootstrap.cc:492] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:24.578235 27450 raft_consensus.cc:2804] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:24.579216 27450 raft_consensus.cc:697] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:24.580549 27450 consensus_queue.cc:237] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.581333 27108 raft_consensus.cc:2468] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:24.583667 27172 leader_election.cc:304] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:24.585184 27453 raft_consensus.cc:2804] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:24.586382 27453 raft_consensus.cc:697] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:24.588742 27407 tablet_bootstrap.cc:492] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:24.588001 27453 consensus_queue.cc:237] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.589289 27407 ts_tablet_manager.cc:1403] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.183s user 0.022s sys 0.001s
I20260807 01:46:24.592765 27407 raft_consensus.cc:359] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.593489 27407 raft_consensus.cc:385] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:24.593770 27407 raft_consensus.cc:740] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:24.601614 27407 consensus_queue.cc:260] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.618386 27407 ts_tablet_manager.cc:1434] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.029s user 0.009s sys 0.000s
I20260807 01:46:24.621032 27407 tablet_bootstrap.cc:492] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:24.635385 27405 tablet_bootstrap.cc:492] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:24.635986 27405 ts_tablet_manager.cc:1403] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.197s user 0.013s sys 0.023s
I20260807 01:46:24.638832 27495 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.644338 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:24.645802 27230 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:24.647264 27337 raft_consensus.cc:2468] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:24.644804 26954 catalog_manager.cc:5697] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:24.655225 26954 catalog_manager.cc:5697] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:24.660750 27405 raft_consensus.cc:359] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.661603 27405 raft_consensus.cc:385] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:24.661830 27405 raft_consensus.cc:740] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:24.662614 27405 consensus_queue.cc:260] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.664070 27404 tablet_bootstrap.cc:654] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:24.664687 27405 ts_tablet_manager.cc:1434] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.028s user 0.007s sys 0.001s
I20260807 01:46:24.667515 27405 tablet_bootstrap.cc:492] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:24.667961 27495 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:24.666286 27407 tablet_bootstrap.cc:654] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:24.673745 27230 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:24.676105 27064 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:24.676832 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:24.677745 27338 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:24.682586 27495 raft_consensus.cc:2804] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:24.683437 27495 raft_consensus.cc:697] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:24.684749 27495 consensus_queue.cc:237] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.702056 27446 consensus_queue.cc:1048] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:24.709189 27404 tablet_bootstrap.cc:492] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:24.717527 27338 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:24.718652 27404 ts_tablet_manager.cc:1403] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.144s user 0.021s sys 0.016s
I20260807 01:46:24.713498 26954 catalog_manager.cc:5697] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:24.724478 27404 raft_consensus.cc:359] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.725395 27404 raft_consensus.cc:385] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:24.725765 27404 raft_consensus.cc:740] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:24.726657 27404 consensus_queue.cc:260] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.727371 27405 tablet_bootstrap.cc:654] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:24.733196 27404 ts_tablet_manager.cc:1434] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.014s user 0.004s sys 0.004s
I20260807 01:46:24.734238 27407 tablet_bootstrap.cc:492] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:24.734686 27404 tablet_bootstrap.cc:492] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:24.734752 27407 ts_tablet_manager.cc:1403] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.114s user 0.021s sys 0.019s
I20260807 01:46:24.746975 27407 raft_consensus.cc:359] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.747792 27407 raft_consensus.cc:385] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:24.748422 27407 raft_consensus.cc:740] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:24.748063 27404 tablet_bootstrap.cc:654] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:24.749291 27407 consensus_queue.cc:260] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.751369 27230 raft_consensus.cc:3060] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:24.752281 27407 ts_tablet_manager.cc:1434] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.017s user 0.000s sys 0.007s
I20260807 01:46:24.764194 27446 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:24.766837 27407 tablet_bootstrap.cc:492] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:24.765054 27446 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.786424 27495 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:24.787747 27495 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.792754 27446 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:24.795238 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:24.795794 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:46:24.798192 27287 leader_election.cc:343] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:24.800582 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:24.799248 27287 leader_election.cc:343] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
W20260807 01:46:24.803681 27064 leader_election.cc:343] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:24.805490 27495 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:24.806275 27287 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:24.809018 27495 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:24.793438 27446 consensus_queue.cc:1048] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260807 01:46:24.811045 27414 raft_consensus.cc:2749] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:24.813341 27414 consensus_queue.cc:1048] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:24.814412 27484 consensus_queue.cc:1048] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:24.814711 27407 tablet_bootstrap.cc:654] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:24.809882 27495 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.829784 27405 tablet_bootstrap.cc:492] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:24.852175 27405 ts_tablet_manager.cc:1403] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.185s user 0.026s sys 0.032s
I20260807 01:46:24.831468 27495 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:24.837626 27494 consensus_queue.cc:1048] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260807 01:46:24.853945 27404 tablet_bootstrap.cc:492] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:24.854478 27404 ts_tablet_manager.cc:1403] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.120s user 0.028s sys 0.058s
I20260807 01:46:24.834489 27484 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:24.834626 27450 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:24.858887 27450 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } }
I20260807 01:46:24.856046 27484 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.869899 27404 raft_consensus.cc:359] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.870690 27404 raft_consensus.cc:385] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:24.871003 27404 raft_consensus.cc:740] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:24.874374 27450 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:46:24.865134 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:24.877174 27404 consensus_queue.cc:260] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:24.865324 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 52761.8us
I20260807 01:46:24.880188 27404 ts_tablet_manager.cc:1434] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.025s user 0.008s sys 0.000s
I20260807 01:46:24.881378 27404 tablet_bootstrap.cc:492] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:24.875823 27484 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:24.887732 27407 tablet_bootstrap.cc:492] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:24.888293 27407 ts_tablet_manager.cc:1403] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.122s user 0.010s sys 0.024s
W20260807 01:46:24.887836 27064 leader_election.cc:343] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:24.891531 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:24.886348 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:24.894862 27484 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:24.895516 27484 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.884341 27495 consensus_queue.cc:1048] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260807 01:46:24.898821 27484 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:24.894029 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:24.889846 27405 raft_consensus.cc:359] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.902046 27405 raft_consensus.cc:385] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:24.902403 27405 raft_consensus.cc:740] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:24.903146 27405 consensus_queue.cc:260] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.913859 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:24.915189 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:24.915901 27063 leader_election.cc:343] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:24.916738 27063 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:24.918282 27495 raft_consensus.cc:2749] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:24.926679 27405 ts_tablet_manager.cc:1434] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.074s user 0.007s sys 0.001s
I20260807 01:46:24.932474 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:24.933626 27105 raft_consensus.cc:2468] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
W20260807 01:46:24.942490 27063 leader_election.cc:343] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:24.943555 27063 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:24.945475 27495 raft_consensus.cc:2749] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:24.952719 27404 tablet_bootstrap.cc:654] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:24.952493 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:24.952832 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:24.955190 27109 raft_consensus.cc:2393] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 1: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:24.956032 27170 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:24.957489 27494 raft_consensus.cc:2749] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:24.961143 27407 raft_consensus.cc:359] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.962095 27407 raft_consensus.cc:385] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:24.962419 27407 raft_consensus.cc:740] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:24.963214 27407 consensus_queue.cc:260] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:24.965946 27407 ts_tablet_manager.cc:1434] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.077s user 0.005s sys 0.001s
I20260807 01:46:24.967218 27407 tablet_bootstrap.cc:492] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:24.967870 27405 tablet_bootstrap.cc:492] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:24.955168 27226 raft_consensus.cc:2468] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:24.976153 27287 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:24.979321 27484 raft_consensus.cc:2804] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:24.979914 27484 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:24.980494 27484 raft_consensus.cc:3060] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:25.003950 27484 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.007803 27484 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:25.034021 27405 tablet_bootstrap.cc:654] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:25.025261 27540 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:25.037820 27540 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.043061 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:46:25.045343 27172 leader_election.cc:343] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:25.047242 27404 tablet_bootstrap.cc:492] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:25.048005 27404 ts_tablet_manager.cc:1403] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.167s user 0.019s sys 0.028s
I20260807 01:46:25.051901 27404 raft_consensus.cc:359] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.052736 27404 raft_consensus.cc:385] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:25.053084 27404 raft_consensus.cc:740] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:25.053920 27404 consensus_queue.cc:260] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.054921 27287 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:25.056414 27404 ts_tablet_manager.cc:1434] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 01:46:25.056447 27414 raft_consensus.cc:3060] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:25.057750 27404 tablet_bootstrap.cc:492] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:25.055936 27484 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:25.061249 27484 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.067952 27407 tablet_bootstrap.cc:654] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:25.082348 27540 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:25.088943 27404 tablet_bootstrap.cc:654] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:25.091033 27495 consensus_queue.cc:1048] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:25.086589 27484 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:25.094528 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:25.096285 27108 raft_consensus.cc:3060] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:25.097457 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:25.098405 27229 raft_consensus.cc:3060] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:25.100900 27405 tablet_bootstrap.cc:492] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:25.101583 27405 ts_tablet_manager.cc:1403] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.134s user 0.012s sys 0.035s
I20260807 01:46:25.102988 27407 tablet_bootstrap.cc:492] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:25.103530 27407 ts_tablet_manager.cc:1403] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.137s user 0.016s sys 0.011s
I20260807 01:46:25.109980 27540 consensus_queue.cc:1048] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:25.111557 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:25.112440 27109 raft_consensus.cc:2468] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:25.129664 27407 raft_consensus.cc:359] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.131341 27407 raft_consensus.cc:385] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:25.131657 27407 raft_consensus.cc:740] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:25.132376 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:25.132531 27407 consensus_queue.cc:260] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.135957 27414 raft_consensus.cc:2749] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:25.138741 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:25.139678 27337 raft_consensus.cc:2468] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:25.139958 27405 raft_consensus.cc:359] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.143174 27407 ts_tablet_manager.cc:1434] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.039s user 0.011s sys 0.000s
I20260807 01:46:25.143828 27404 tablet_bootstrap.cc:492] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:25.140691 27405 raft_consensus.cc:385] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:25.144398 27404 ts_tablet_manager.cc:1403] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.087s user 0.020s sys 0.023s
I20260807 01:46:25.147470 27405 raft_consensus.cc:740] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:25.148231 27170 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:25.149057 27407 tablet_bootstrap.cc:492] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
W20260807 01:46:25.149262 27287 leader_election.cc:343] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:25.150408 27450 raft_consensus.cc:2804] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:25.150913 27450 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:25.151328 27450 raft_consensus.cc:3060] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:25.144917 27229 raft_consensus.cc:2468] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:25.146934 27108 raft_consensus.cc:2468] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:25.154341 27405 consensus_queue.cc:260] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.156217 27287 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:25.157454 27484 raft_consensus.cc:2804] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:25.157967 27484 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:25.158468 27484 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:25.155500 27540 consensus_queue.cc:1048] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260807 01:46:25.165125 27450 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.171615 27405 ts_tablet_manager.cc:1434] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.070s user 0.006s sys 0.002s
I20260807 01:46:25.172935 27405 tablet_bootstrap.cc:492] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:25.173544 27407 tablet_bootstrap.cc:654] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:25.175911 27404 raft_consensus.cc:359] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.176649 27404 raft_consensus.cc:385] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:25.176934 27404 raft_consensus.cc:740] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:25.177662 27404 consensus_queue.cc:260] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.182848 27404 ts_tablet_manager.cc:1434] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.035s user 0.006s sys 0.001s
I20260807 01:46:25.185290 27287 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:25.186614 27530 raft_consensus.cc:2804] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:25.187445 27530 raft_consensus.cc:697] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:25.188656 27404 tablet_bootstrap.cc:492] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:25.189613 27530 consensus_queue.cc:237] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.190704 27450 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:25.195349 27540 consensus_queue.cc:1048] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:25.199021 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:25.199812 27337 raft_consensus.cc:3060] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:25.231037 27484 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.207285 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:25.279541 27405 tablet_bootstrap.cc:654] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:25.285727 27484 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:25.288431 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:25.288817 27414 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:25.288431 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:25.289741 27414 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:25.295622 27287 leader_election.cc:343] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:25.292501 27108 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:25.307808 27540 consensus_queue.cc:1048] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260807 01:46:25.314916 27172 leader_election.cc:343] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:25.316732 27494 consensus_queue.cc:1048] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:25.385840 27461 consensus_queue.cc:1048] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:25.385599 27337 raft_consensus.cc:2468] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:25.400429 27461 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:25.401185 27461 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.380846 27414 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:25.406611 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:25.410037 27407 tablet_bootstrap.cc:492] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
W20260807 01:46:25.415239 27064 leader_election.cc:343] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:25.420807 27407 ts_tablet_manager.cc:1403] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.272s user 0.023s sys 0.017s
I20260807 01:46:25.421386 27405 tablet_bootstrap.cc:492] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:25.421902 27405 ts_tablet_manager.cc:1403] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.249s user 0.014s sys 0.003s
I20260807 01:46:25.423312 27404 tablet_bootstrap.cc:654] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:25.429204 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:25.430272 27461 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:46:25.434880 27287 leader_election.cc:343] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:25.435897 27170 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:25.438194 27495 consensus_queue.cc:1048] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260807 01:46:25.438647 27461 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:25.439569 27461 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.441747 27540 raft_consensus.cc:2804] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:25.439821 26950 catalog_manager.cc:5697] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:25.442703 27540 raft_consensus.cc:697] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:25.443961 27540 consensus_queue.cc:237] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.445379 27461 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:25.454008 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:46:25.454523 27036 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 01:46:25.447068 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:25.459448 27405 raft_consensus.cc:359] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.460260 27405 raft_consensus.cc:385] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:25.460563 27405 raft_consensus.cc:740] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:25.463953 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:46:25.468372 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 55813.4us
I20260807 01:46:25.473229 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:25.470849 27287 leader_election.cc:343] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:25.478921 27287 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:25.480157 27484 raft_consensus.cc:2749] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:25.479267 27108 raft_consensus.cc:2468] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:25.485373 27405 consensus_queue.cc:260] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.490689 27405 ts_tablet_manager.cc:1434] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.068s user 0.006s sys 0.005s
I20260807 01:46:25.493304 27405 tablet_bootstrap.cc:492] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:25.496934 27287 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:25.498081 27484 raft_consensus.cc:2804] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:25.498682 27484 raft_consensus.cc:697] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
W20260807 01:46:25.498903 27063 leader_election.cc:343] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:25.499416 27063 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:25.501456 27407 raft_consensus.cc:359] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.501626 27572 raft_consensus.cc:2749] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:25.499822 27484 consensus_queue.cc:237] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:25.496179 27064 leader_election.cc:343] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:25.501976 27407 raft_consensus.cc:385] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:25.507037 27407 raft_consensus.cc:740] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:25.509194 27407 consensus_queue.cc:260] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:25.510620 27063 leader_election.cc:343] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:25.516258 27407 ts_tablet_manager.cc:1434] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.095s user 0.009s sys 0.000s
I20260807 01:46:25.513567 27063 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:25.517763 27461 raft_consensus.cc:2749] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:25.517609 27407 tablet_bootstrap.cc:492] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:25.532191 27405 tablet_bootstrap.cc:654] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:25.547899 27404 tablet_bootstrap.cc:492] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:25.548426 27404 ts_tablet_manager.cc:1403] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.364s user 0.021s sys 0.027s
I20260807 01:46:25.553639 27404 raft_consensus.cc:359] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.554416 27404 raft_consensus.cc:385] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:25.554706 27404 raft_consensus.cc:740] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:25.555594 27407 tablet_bootstrap.cc:654] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:25.558645 27404 consensus_queue.cc:260] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.560909 27540 raft_consensus.cc:493] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:25.561841 27404 ts_tablet_manager.cc:1434] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.013s user 0.005s sys 0.005s
I20260807 01:46:25.561725 27540 raft_consensus.cc:515] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.563333 27404 tablet_bootstrap.cc:492] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:25.571436 27540 leader_election.cc:290] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:25.574748 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:25.574851 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:25.575773 27336 raft_consensus.cc:2468] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
W20260807 01:46:25.576850 27172 leader_election.cc:343] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:25.577877 27170 leader_election.cc:304] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:25.579247 27540 raft_consensus.cc:2804] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:25.580866 27540 raft_consensus.cc:493] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:25.581605 27540 raft_consensus.cc:3060] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:25.588364 27405 tablet_bootstrap.cc:492] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:25.588986 27405 ts_tablet_manager.cc:1403] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.096s user 0.027s sys 0.013s
I20260807 01:46:25.590262 27404 tablet_bootstrap.cc:654] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:25.592828 27405 raft_consensus.cc:359] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.593719 27405 raft_consensus.cc:385] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:25.593997 27405 raft_consensus.cc:740] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:25.594758 27405 consensus_queue.cc:260] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.599138 26952 catalog_manager.cc:5697] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:25.610095 27540 raft_consensus.cc:515] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.620450 27405 ts_tablet_manager.cc:1434] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.031s user 0.006s sys 0.003s
I20260807 01:46:25.621778 27405 tablet_bootstrap.cc:492] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:25.623445 27540 leader_election.cc:290] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:25.625290 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:25.625694 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
W20260807 01:46:25.628182 27172 leader_election.cc:343] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:25.628657 27407 tablet_bootstrap.cc:492] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:25.628374 27336 raft_consensus.cc:3060] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:25.629201 27407 ts_tablet_manager.cc:1403] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.112s user 0.025s sys 0.030s
I20260807 01:46:25.632110 26954 catalog_manager.cc:5697] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:25.635941 27404 tablet_bootstrap.cc:492] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:25.636507 27404 ts_tablet_manager.cc:1403] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.073s user 0.013s sys 0.026s
I20260807 01:46:25.639230 27405 tablet_bootstrap.cc:654] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:25.639513 27404 raft_consensus.cc:359] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.640226 27404 raft_consensus.cc:385] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:25.640499 27404 raft_consensus.cc:740] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:25.641341 27404 consensus_queue.cc:260] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.643464 27404 ts_tablet_manager.cc:1434] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.007s user 0.004s sys 0.002s
I20260807 01:46:25.651140 27404 tablet_bootstrap.cc:492] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:25.653296 27407 raft_consensus.cc:359] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.654301 27407 raft_consensus.cc:385] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:25.659844 27407 raft_consensus.cc:740] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:25.663923 27407 consensus_queue.cc:260] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.665689 27404 tablet_bootstrap.cc:654] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:25.666726 27407 ts_tablet_manager.cc:1434] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.037s user 0.007s sys 0.000s
I20260807 01:46:25.649376 27540 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:25.667426 27540 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.670462 27540 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:25.662796 27336 raft_consensus.cc:2468] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:25.671844 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:25.672538 27337 raft_consensus.cc:2468] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:25.673137 27407 tablet_bootstrap.cc:492] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:25.674084 27170 leader_election.cc:304] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:25.674919 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:25.675336 27170 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:25.676112 27540 raft_consensus.cc:2804] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:25.676131 27450 raft_consensus.cc:2804] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:25.676594 27540 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:25.676973 27540 raft_consensus.cc:3060] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:25.676965 27450 raft_consensus.cc:697] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:25.680433 27108 raft_consensus.cc:2468] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:25.682003 27450 consensus_queue.cc:237] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.694952 27407 tablet_bootstrap.cc:654] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:25.697255 27405 tablet_bootstrap.cc:492] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:25.700059 27405 ts_tablet_manager.cc:1403] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.079s user 0.010s sys 0.037s
I20260807 01:46:25.700419 27540 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.703835 27405 raft_consensus.cc:359] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.704378 27405 raft_consensus.cc:385] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:25.704663 27405 raft_consensus.cc:740] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:25.704596 26952 catalog_manager.cc:5697] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:25.705910 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:25.706703 27108 raft_consensus.cc:3060] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:25.706791 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:25.707580 27337 raft_consensus.cc:3060] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:25.710515 27540 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:25.711700 27405 consensus_queue.cc:260] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.720618 27405 ts_tablet_manager.cc:1434] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.020s user 0.003s sys 0.005s
I20260807 01:46:25.717598 27108 raft_consensus.cc:2468] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:25.723907 27404 tablet_bootstrap.cc:492] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:25.724419 27404 ts_tablet_manager.cc:1403] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.074s user 0.015s sys 0.001s
I20260807 01:46:25.724455 27172 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:25.725011 27337 raft_consensus.cc:2468] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:25.726034 27540 raft_consensus.cc:2804] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:25.726599 27540 raft_consensus.cc:697] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:25.727833 27407 tablet_bootstrap.cc:492] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:25.728365 27407 ts_tablet_manager.cc:1403] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.056s user 0.017s sys 0.005s
I20260807 01:46:25.727749 27540 consensus_queue.cc:237] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.722970 27405 tablet_bootstrap.cc:492] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:25.734375 27407 raft_consensus.cc:359] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.735253 27407 raft_consensus.cc:385] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:25.735531 27407 raft_consensus.cc:740] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:25.736346 27407 consensus_queue.cc:260] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.738478 27407 ts_tablet_manager.cc:1434] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.010s user 0.007s sys 0.003s
I20260807 01:46:25.733561 27404 raft_consensus.cc:359] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.739485 27404 raft_consensus.cc:385] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:25.739554 27407 tablet_bootstrap.cc:492] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:25.739903 27404 raft_consensus.cc:740] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:25.743000 27404 consensus_queue.cc:260] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.746317 27404 ts_tablet_manager.cc:1434] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.016s user 0.003s sys 0.005s
I20260807 01:46:25.748778 27404 tablet_bootstrap.cc:492] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:25.756454 27405 tablet_bootstrap.cc:654] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:25.762010 27404 tablet_bootstrap.cc:654] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:25.762742 27407 tablet_bootstrap.cc:654] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:25.763127 26952 catalog_manager.cc:5697] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:25.770386 27407 tablet_bootstrap.cc:492] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:25.770831 27407 ts_tablet_manager.cc:1403] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.031s user 0.005s sys 0.012s
I20260807 01:46:25.771988 27484 consensus_queue.cc:1048] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:25.777632 27407 raft_consensus.cc:359] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.778728 27405 tablet_bootstrap.cc:492] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:25.779889 27405 ts_tablet_manager.cc:1403] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.057s user 0.017s sys 0.001s
I20260807 01:46:25.778846 27484 consensus_queue.cc:1048] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:25.782603 27405 raft_consensus.cc:359] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.785858 27405 raft_consensus.cc:385] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:25.786193 27405 raft_consensus.cc:740] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:25.787094 27405 consensus_queue.cc:260] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.787771 27407 raft_consensus.cc:385] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:25.788091 27407 raft_consensus.cc:740] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:25.788654 27540 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:25.788849 27407 consensus_queue.cc:260] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.789768 27405 ts_tablet_manager.cc:1434] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260807 01:46:25.790736 27405 tablet_bootstrap.cc:492] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:25.791157 27407 ts_tablet_manager.cc:1434] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.020s user 0.007s sys 0.000s
I20260807 01:46:25.792191 27407 tablet_bootstrap.cc:492] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:25.793282 27540 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.817472 27540 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:25.818321 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:25.819258 27109 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:25.821532 27404 tablet_bootstrap.cc:492] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:25.822088 27404 ts_tablet_manager.cc:1403] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.074s user 0.010s sys 0.008s
I20260807 01:46:25.822443 27172 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:25.823740 27540 raft_consensus.cc:2804] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:25.824218 27540 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:25.824788 27540 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:25.830389 27404 raft_consensus.cc:359] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.831209 27404 raft_consensus.cc:385] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:25.831485 27404 raft_consensus.cc:740] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:25.832681 27404 consensus_queue.cc:260] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.834271 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:25.836009 27404 ts_tablet_manager.cc:1434] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.014s user 0.005s sys 0.003s
I20260807 01:46:25.837198 27404 tablet_bootstrap.cc:492] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
W20260807 01:46:25.840036 27170 leader_election.cc:343] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:25.839614 27540 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.843061 27540 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:25.844152 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:25.844983 27108 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:25.848055 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:25.851030 27495 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:25.851979 27484 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:25.851828 27495 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.852591 27484 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.854259 27405 tablet_bootstrap.cc:654] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:25.855960 27484 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:25.856902 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:25.858856 27064 leader_election.cc:343] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
W20260807 01:46:25.860536 27170 leader_election.cc:343] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:25.862746 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:25.865666 27495 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:25.868439 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:46:25.871232 27287 leader_election.cc:343] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:25.872126 27407 tablet_bootstrap.cc:654] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:25.878109 27404 tablet_bootstrap.cc:654] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:25.887590 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:25.888482 27336 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
W20260807 01:46:25.889787 27287 leader_election.cc:343] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:25.891352 27450 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:25.893188 27450 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.897536 27450 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:25.888095 27108 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:25.898253 27287 leader_election.cc:304] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:25.899041 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:25.899855 27172 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:46:25.901095 27450 raft_consensus.cc:2804] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:25.902716 27484 raft_consensus.cc:2749] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:25.903913 27063 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
W20260807 01:46:25.903731 27172 leader_election.cc:343] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:25.906348 27572 raft_consensus.cc:2804] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:25.906946 27572 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:25.908345 27572 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:25.911267 27404 tablet_bootstrap.cc:492] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:25.911872 27404 ts_tablet_manager.cc:1403] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.075s user 0.010s sys 0.014s
I20260807 01:46:25.903096 27450 raft_consensus.cc:697] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:25.914733 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:25.916953 27336 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:25.919752 27450 consensus_queue.cc:237] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.926023 27407 tablet_bootstrap.cc:492] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:25.925837 27404 raft_consensus.cc:359] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.926718 27407 ts_tablet_manager.cc:1403] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.135s user 0.021s sys 0.009s
I20260807 01:46:25.926724 27404 raft_consensus.cc:385] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:25.928213 27404 raft_consensus.cc:740] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:25.927695 27170 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:25.930768 27494 raft_consensus.cc:2804] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:25.931280 27494 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:25.931955 27494 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:25.931841 27404 consensus_queue.cc:260] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.934599 27407 raft_consensus.cc:359] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.935633 27407 raft_consensus.cc:385] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:25.935909 27407 raft_consensus.cc:740] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:25.936600 27407 consensus_queue.cc:260] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.936585 27572 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.941728 27404 ts_tablet_manager.cc:1434] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.030s user 0.004s sys 0.007s
I20260807 01:46:25.942878 27404 tablet_bootstrap.cc:492] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:25.944396 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
W20260807 01:46:25.946574 27064 leader_election.cc:343] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:25.948268 27407 ts_tablet_manager.cc:1434] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.021s user 0.005s sys 0.005s
I20260807 01:46:25.948390 27540 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:25.949213 27572 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:25.949373 27540 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.950194 27407 tablet_bootstrap.cc:492] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:25.958618 27404 tablet_bootstrap.cc:654] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:25.959510 27572 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:25.954805 27540 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:25.960554 27572 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.965430 27572 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:25.965852 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:25.966562 27337 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:25.969954 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:25.970963 27108 raft_consensus.cc:2468] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:25.966897 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:25.972798 27333 raft_consensus.cc:2468] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:25.974588 27572 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:25.974673 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:25.976743 27540 consensus_queue.cc:1048] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:25.977103 27172 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:25.972859 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:25.985945 27453 raft_consensus.cc:2804] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:25.981863 27572 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.988077 27453 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:25.990824 27453 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:25.991272 27405 tablet_bootstrap.cc:492] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:25.991781 27405 ts_tablet_manager.cc:1403] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.201s user 0.009s sys 0.016s
W20260807 01:46:25.989773 27064 leader_election.cc:343] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:25.997201 27405 raft_consensus.cc:359] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:25.997956 27405 raft_consensus.cc:385] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:25.998296 27405 raft_consensus.cc:740] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:25.999094 27405 consensus_queue.cc:260] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.003185 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:25.994930 27407 tablet_bootstrap.cc:654] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
W20260807 01:46:26.007181 27063 leader_election.cc:343] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:26.008291 27063 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:26.010432 27405 ts_tablet_manager.cc:1434] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.018s user 0.006s sys 0.003s
I20260807 01:46:26.017741 27405 tablet_bootstrap.cc:492] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:26.011704 27495 raft_consensus.cc:2749] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:26.011974 27572 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:26.023451 27108 raft_consensus.cc:3060] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:26.005425 27453 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.034484 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:26.035415 27104 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:26.035274 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:26.032039 26952 catalog_manager.cc:5697] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:26.042074 27337 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:26.042526 27453 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:26.044317 27450 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:26.046900 27450 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.050141 27494 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.051167 27063 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:26.054677 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:26.058320 27063 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:26.060564 27572 raft_consensus.cc:2749] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:26.063423 27495 raft_consensus.cc:2804] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:26.064610 27495 raft_consensus.cc:697] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:26.055774 27337 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:26.066241 27495 consensus_queue.cc:237] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.055996 27494 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:26.090327 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:26.105460 27414 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:26.106287 27414 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.110257 27414 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:26.126858 27405 tablet_bootstrap.cc:654] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:26.126557 27104 raft_consensus.cc:2468] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
W20260807 01:46:26.127542 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 69496.6us
I20260807 01:46:26.126442 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:26.077365 27540 consensus_queue.cc:1048] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.034s
I20260807 01:46:26.129897 27333 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:26.137184 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:26.140244 27407 tablet_bootstrap.cc:492] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:26.140771 27407 ts_tablet_manager.cc:1403] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.191s user 0.017s sys 0.017s
W20260807 01:46:26.135376 27172 leader_election.cc:343] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:26.143558 27404 tablet_bootstrap.cc:492] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:26.143443 27337 raft_consensus.cc:2468] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:26.144150 27404 ts_tablet_manager.cc:1403] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.201s user 0.025s sys 0.015s
I20260807 01:46:26.144711 27172 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:26.145649 27494 raft_consensus.cc:2804] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:26.146128 27494 raft_consensus.cc:697] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:26.147022 27494 consensus_queue.cc:237] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.155234 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:26.155978 27105 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:26.156163 27450 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:26.159235 27407 raft_consensus.cc:359] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.159595 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:26.159986 27407 raft_consensus.cc:385] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:26.160297 27407 raft_consensus.cc:740] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:26.160979 27407 consensus_queue.cc:260] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:26.161537 27287 leader_election.cc:343] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:26.162575 27404 raft_consensus.cc:359] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.162997 27337 raft_consensus.cc:2393] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 1: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:26.163280 27404 raft_consensus.cc:385] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:26.163616 27404 raft_consensus.cc:740] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:26.165081 27404 consensus_queue.cc:260] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.168118 27404 ts_tablet_manager.cc:1434] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.024s user 0.006s sys 0.002s
I20260807 01:46:26.168655 27407 ts_tablet_manager.cc:1434] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.028s user 0.004s sys 0.003s
I20260807 01:46:26.169694 27407 tablet_bootstrap.cc:492] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:26.174296 27404 tablet_bootstrap.cc:492] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:26.231648 27407 tablet_bootstrap.cc:654] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:26.244320 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:46:26.246508 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 54867us
I20260807 01:46:26.263635 26954 catalog_manager.cc:5697] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:26.266311 27287 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:26.267649 27484 raft_consensus.cc:2804] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:26.268270 27484 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:26.251617 27572 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:26.270694 27484 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:26.273213 27414 consensus_queue.cc:1048] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260807 01:46:26.277853 27172 consensus_peers.cc:597] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f -> Peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Couldn't send request to peer 3262a9d183914b699227fdba0ca5bed3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:26.267429 26950 catalog_manager.cc:5697] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:26.280586 27572 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.279012 27333 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:26.252194 27494 consensus_queue.cc:1048] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260807 01:46:26.288436 27172 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:26.289330 27453 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:26.293188 27405 tablet_bootstrap.cc:492] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:26.299530 27405 ts_tablet_manager.cc:1403] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.282s user 0.014s sys 0.032s
I20260807 01:46:26.297024 27572 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:26.298669 27484 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.300765 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:26.295495 27287 consensus_peers.cc:597] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade -> Peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Couldn't send request to peer 3478729505b046c2a0051c2d0130873f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:26.302358 27484 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:46:26.307552 27064 leader_election.cc:343] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:26.314921 27494 consensus_queue.cc:1048] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:26.325647 27453 raft_consensus.cc:2749] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:26.367559 27405 raft_consensus.cc:359] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.368350 27404 tablet_bootstrap.cc:654] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:26.369580 27405 raft_consensus.cc:385] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:26.370252 27405 raft_consensus.cc:740] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:26.374969 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:26.375053 27494 consensus_queue.cc:1048] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:26.375794 27109 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:26.379055 27407 tablet_bootstrap.cc:492] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:26.379629 27407 ts_tablet_manager.cc:1403] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.210s user 0.007s sys 0.011s
I20260807 01:46:26.402997 27405 consensus_queue.cc:260] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.409289 27109 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:26.407482 27407 raft_consensus.cc:359] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.412753 27407 raft_consensus.cc:385] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:26.413105 27407 raft_consensus.cc:740] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:26.414697 27407 consensus_queue.cc:260] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.431841 27407 ts_tablet_manager.cc:1434] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.052s user 0.008s sys 0.003s
I20260807 01:46:26.439512 27407 tablet_bootstrap.cc:492] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:26.448488 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:26.450228 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:26.481793 27405 ts_tablet_manager.cc:1434] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.182s user 0.015s sys 0.003s
I20260807 01:46:26.467276 27484 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:26.482223 27170 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:26.483320 27484 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.484767 27405 tablet_bootstrap.cc:492] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:26.484975 27453 raft_consensus.cc:2804] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:26.485635 27453 raft_consensus.cc:697] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:26.486601 27453 consensus_queue.cc:237] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.522980 27336 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:26.523550 27404 tablet_bootstrap.cc:492] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:26.524401 27404 ts_tablet_manager.cc:1403] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.350s user 0.040s sys 0.043s
W20260807 01:46:26.540721 27625 log.cc:927] Time spent T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.092s user 0.000s sys 0.006s
I20260807 01:46:26.519273 27287 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:26.545837 27484 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:26.552657 27414 raft_consensus.cc:2804] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election won for term 1
W20260807 01:46:26.552930 27063 leader_election.cc:343] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:26.553771 27063 leader_election.cc:304] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:26.546057 27404 raft_consensus.cc:359] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.564690 27404 raft_consensus.cc:385] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:26.565119 27404 raft_consensus.cc:740] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:26.566196 27572 raft_consensus.cc:2749] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:26.564028 27414 raft_consensus.cc:697] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:26.572816 27404 consensus_queue.cc:260] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.574798 27572 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:26.579519 27572 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.590998 27494 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:26.592311 27494 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.597944 27404 ts_tablet_manager.cc:1434] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.073s user 0.005s sys 0.005s
I20260807 01:46:26.599650 27404 tablet_bootstrap.cc:492] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:26.602886 27572 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:26.607488 27494 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:26.614578 27405 tablet_bootstrap.cc:654] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
W20260807 01:46:26.614823 27624 log.cc:927] Time spent T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.106s user 0.001s sys 0.007s
I20260807 01:46:26.623296 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:26.624346 27227 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
I20260807 01:46:26.628242 27404 tablet_bootstrap.cc:654] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:26.629071 27064 leader_election.cc:304] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:26.635982 27572 raft_consensus.cc:2804] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:26.636757 27572 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 01:46:26.637519 27036 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 01:46:26.639144 27407 tablet_bootstrap.cc:654] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:26.637691 27572 raft_consensus.cc:3060] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:26.651815 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:26.653270 27494 consensus_queue.cc:1048] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:26.657928 26950 catalog_manager.cc:5697] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:26.644371 27414 consensus_queue.cc:237] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.663308 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:26.666636 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:26.674137 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:26.675067 27337 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
I20260807 01:46:26.677750 27453 consensus_queue.cc:1048] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260807 01:46:26.679615 27287 leader_election.cc:343] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:26.691035 27572 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.693472 27108 raft_consensus.cc:2393] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 1: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:26.723229 27572 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:26.732234 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:26.733140 27226 raft_consensus.cc:3060] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:26.737494 27461 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:26.738200 27461 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.742084 27572 consensus_queue.cc:1048] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:26.744869 27287 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:26.749185 26952 catalog_manager.cc:5697] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:26.748564 27453 consensus_queue.cc:1048] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260807 01:46:26.754762 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:26.759510 27414 raft_consensus.cc:2749] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:26.759922 27226 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:26.750723 27461 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:26.762344 27064 leader_election.cc:304] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:26.764140 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:26.768538 27336 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:26.764137 27404 tablet_bootstrap.cc:492] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:26.769565 27461 raft_consensus.cc:2804] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:26.770295 27404 ts_tablet_manager.cc:1403] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.171s user 0.019s sys 0.021s
I20260807 01:46:26.770493 27461 raft_consensus.cc:697] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:26.765317 27405 tablet_bootstrap.cc:492] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:26.771642 27405 ts_tablet_manager.cc:1403] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.287s user 0.025s sys 0.021s
I20260807 01:46:26.767472 27407 tablet_bootstrap.cc:492] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:26.772429 27407 ts_tablet_manager.cc:1403] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.339s user 0.024s sys 0.027s
I20260807 01:46:26.777866 27405 raft_consensus.cc:359] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.779500 27405 raft_consensus.cc:385] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:26.779798 27405 raft_consensus.cc:740] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:26.802083 27404 raft_consensus.cc:359] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.803337 27404 raft_consensus.cc:385] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:26.786955 27461 consensus_queue.cc:237] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.803742 27404 raft_consensus.cc:740] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:26.788801 27405 consensus_queue.cc:260] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.798569 27414 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:26.808157 27404 consensus_queue.cc:260] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.808380 27414 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.808820 27407 raft_consensus.cc:359] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.813540 27170 leader_election.cc:304] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:26.814170 27407 raft_consensus.cc:385] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:26.814623 27407 raft_consensus.cc:740] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:26.814782 27405 ts_tablet_manager.cc:1434] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.043s user 0.011s sys 0.005s
I20260807 01:46:26.818522 27407 consensus_queue.cc:260] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.819259 27622 consensus_queue.cc:1048] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260807 01:46:26.822305 27494 raft_consensus.cc:2764] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260807 01:46:26.823414 27405 tablet_bootstrap.cc:492] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:26.824241 27407 ts_tablet_manager.cc:1434] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.051s user 0.010s sys 0.000s
I20260807 01:46:26.826648 27407 tablet_bootstrap.cc:492] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:26.827196 27404 ts_tablet_manager.cc:1434] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.056s user 0.009s sys 0.004s
I20260807 01:46:26.829357 27404 tablet_bootstrap.cc:492] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:26.833505 27414 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:26.848443 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:26.849438 27331 raft_consensus.cc:3060] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:26.863749 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:26.865477 27336 raft_consensus.cc:2393] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 1: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:26.867738 26954 catalog_manager.cc:5697] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:26.869377 27104 raft_consensus.cc:3060] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:26.890898 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:26.892468 27108 raft_consensus.cc:2468] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:26.898484 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:26.899215 27229 raft_consensus.cc:2468] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:26.903504 27461 consensus_queue.cc:1048] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:26.910100 27404 tablet_bootstrap.cc:654] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:26.914132 27063 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:26.919754 27461 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:26.940100 27407 tablet_bootstrap.cc:654] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:26.942921 27405 tablet_bootstrap.cc:654] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:26.963739 27404 tablet_bootstrap.cc:492] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:26.964264 27404 ts_tablet_manager.cc:1403] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.135s user 0.001s sys 0.023s
I20260807 01:46:26.960000 27461 raft_consensus.cc:2749] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:26.978570 27404 raft_consensus.cc:359] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.979342 27404 raft_consensus.cc:385] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:26.979643 27404 raft_consensus.cc:740] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:26.980340 27404 consensus_queue.cc:260] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:26.983314 27404 ts_tablet_manager.cc:1434] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.019s user 0.008s sys 0.000s
I20260807 01:46:26.984386 27404 tablet_bootstrap.cc:492] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:26.983995 27287 leader_election.cc:304] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:26.992174 27414 raft_consensus.cc:2804] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:26.992848 27414 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:26.993459 27414 raft_consensus.cc:3060] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:27.006349 27450 consensus_queue.cc:1048] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:26.979060 27331 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:27.045598 27484 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:27.049386 27484 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.053576 27484 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:27.057392 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:27.059048 27633 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:27.060407 27633 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.057626 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:27.064000 27287 leader_election.cc:343] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
W20260807 01:46:27.065485 27287 leader_election.cc:343] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:27.065972 27287 leader_election.cc:304] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:27.068236 27629 raft_consensus.cc:2749] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:27.070923 27404 tablet_bootstrap.cc:654] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:27.074951 27633 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:27.076884 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:27.077715 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:27.079324 27407 tablet_bootstrap.cc:492] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:27.079814 27407 ts_tablet_manager.cc:1403] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.253s user 0.040s sys 0.010s
I20260807 01:46:27.066994 27414 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:27.081838 27287 leader_election.cc:343] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
W20260807 01:46:27.082612 27287 leader_election.cc:343] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:27.082921 27287 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:27.083266 27414 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:27.083567 27629 raft_consensus.cc:2749] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:27.085352 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:27.085459 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:27.087107 27229 raft_consensus.cc:3060] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:27.086797 27110 raft_consensus.cc:3060] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:27.109001 27405 tablet_bootstrap.cc:492] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:27.114594 27405 ts_tablet_manager.cc:1403] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.291s user 0.010s sys 0.019s
I20260807 01:46:27.119556 27407 raft_consensus.cc:359] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.120405 27407 raft_consensus.cc:385] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:27.120698 27407 raft_consensus.cc:740] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:27.121435 27407 consensus_queue.cc:260] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.124943 27405 raft_consensus.cc:359] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.125772 27405 raft_consensus.cc:385] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:27.129879 27110 raft_consensus.cc:2468] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:27.131723 27407 ts_tablet_manager.cc:1434] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.052s user 0.010s sys 0.003s
I20260807 01:46:27.134284 27407 tablet_bootstrap.cc:492] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:27.136664 27405 raft_consensus.cc:740] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:27.138906 27404 tablet_bootstrap.cc:492] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:27.139408 27404 ts_tablet_manager.cc:1403] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.155s user 0.019s sys 0.024s
I20260807 01:46:27.140852 27287 leader_election.cc:304] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:27.144387 27229 raft_consensus.cc:2468] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:27.144886 27629 raft_consensus.cc:2804] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:27.144963 27622 consensus_queue.cc:1048] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:27.145974 27629 raft_consensus.cc:697] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:27.140092 27405 consensus_queue.cc:260] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.148491 27629 consensus_queue.cc:237] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.153224 27633 consensus_queue.cc:1048] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:27.157172 27405 ts_tablet_manager.cc:1434] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.042s user 0.003s sys 0.006s
I20260807 01:46:27.165104 27405 tablet_bootstrap.cc:492] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:27.167984 27404 raft_consensus.cc:359] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.168728 27404 raft_consensus.cc:385] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:27.168998 27404 raft_consensus.cc:740] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:27.171000 27404 consensus_queue.cc:260] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.187335 27229 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:27.191644 27404 ts_tablet_manager.cc:1434] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.052s user 0.004s sys 0.005s
I20260807 01:46:27.203960 27404 tablet_bootstrap.cc:492] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:27.219969 27622 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:27.220038 26952 catalog_manager.cc:5697] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:27.220691 27622 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.227260 27407 tablet_bootstrap.cc:654] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:27.229619 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:27.265892 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:27.269500 27170 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:27.271720 27494 raft_consensus.cc:2749] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:27.282284 27405 tablet_bootstrap.cc:654] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:27.284721 27629 consensus_queue.cc:1048] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:27.263042 27622 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:27.289137 27404 tablet_bootstrap.cc:654] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:27.350245 27572 consensus_queue.cc:1048] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:27.361260 27404 tablet_bootstrap.cc:492] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:27.363376 27405 tablet_bootstrap.cc:492] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:27.363976 27405 ts_tablet_manager.cc:1403] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.199s user 0.021s sys 0.022s
I20260807 01:46:27.365870 27407 tablet_bootstrap.cc:492] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:27.368943 27407 ts_tablet_manager.cc:1403] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.235s user 0.015s sys 0.019s
I20260807 01:46:27.362699 27404 ts_tablet_manager.cc:1403] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.170s user 0.009s sys 0.021s
I20260807 01:46:27.378721 27405 raft_consensus.cc:359] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.381836 27405 raft_consensus.cc:385] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:27.382277 27405 raft_consensus.cc:740] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:27.382655 27404 raft_consensus.cc:359] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.383328 27404 raft_consensus.cc:385] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:27.383750 27404 raft_consensus.cc:740] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:27.383204 27405 consensus_queue.cc:260] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.384460 27404 consensus_queue.cc:260] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.386703 27405 ts_tablet_manager.cc:1434] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.020s user 0.006s sys 0.003s
I20260807 01:46:27.387276 27404 ts_tablet_manager.cc:1434] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.017s user 0.009s sys 0.001s
I20260807 01:46:27.387132 27407 raft_consensus.cc:359] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.388002 27407 raft_consensus.cc:385] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:27.388352 27407 raft_consensus.cc:740] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:27.388737 27404 tablet_bootstrap.cc:492] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:27.389295 27407 consensus_queue.cc:260] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.390105 27572 consensus_queue.cc:1048] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:27.392370 27405 tablet_bootstrap.cc:492] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:27.393337 27407 ts_tablet_manager.cc:1434] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.024s user 0.007s sys 0.001s
I20260807 01:46:27.397591 27407 tablet_bootstrap.cc:492] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:27.404430 27404 tablet_bootstrap.cc:654] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:27.462270 27405 tablet_bootstrap.cc:654] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:27.473196 27407 tablet_bootstrap.cc:654] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:27.493762 27404 tablet_bootstrap.cc:492] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:27.494290 27404 ts_tablet_manager.cc:1403] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.106s user 0.005s sys 0.025s
I20260807 01:46:27.502030 27404 raft_consensus.cc:359] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.503104 27404 raft_consensus.cc:385] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:27.503417 27404 raft_consensus.cc:740] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:27.504235 27404 consensus_queue.cc:260] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.506686 27404 ts_tablet_manager.cc:1434] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.012s user 0.008s sys 0.002s
I20260807 01:46:27.509563 27404 tablet_bootstrap.cc:492] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:27.515560 27405 tablet_bootstrap.cc:492] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:27.516243 27405 ts_tablet_manager.cc:1403] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.124s user 0.013s sys 0.021s
I20260807 01:46:27.517968 27407 tablet_bootstrap.cc:492] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:27.519141 27407 ts_tablet_manager.cc:1403] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.122s user 0.011s sys 0.022s
I20260807 01:46:27.522017 27405 raft_consensus.cc:359] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.526247 27405 raft_consensus.cc:385] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:27.526698 27405 raft_consensus.cc:740] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:27.527756 27405 consensus_queue.cc:260] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.532073 27405 ts_tablet_manager.cc:1434] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.015s user 0.008s sys 0.000s
I20260807 01:46:27.530753 27404 tablet_bootstrap.cc:654] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:27.539338 27405 tablet_bootstrap.cc:492] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:27.522346 27407 raft_consensus.cc:359] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.544649 27407 raft_consensus.cc:385] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:27.545006 27407 raft_consensus.cc:740] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:27.545879 27407 consensus_queue.cc:260] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.553190 27407 ts_tablet_manager.cc:1434] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.034s user 0.003s sys 0.003s
I20260807 01:46:27.553280 27229 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:27.554392 27407 tablet_bootstrap.cc:492] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:27.559680 27461 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:27.563282 27461 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.570267 27461 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:27.570712 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:27.571748 27333 raft_consensus.cc:2468] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
I20260807 01:46:27.573433 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:27.577580 27405 tablet_bootstrap.cc:654] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:27.588481 27063 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:27.590268 27461 raft_consensus.cc:2804] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260807 01:46:27.591327 27064 leader_election.cc:343] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:27.592087 27461 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:27.593212 27461 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:27.597585 27404 tablet_bootstrap.cc:492] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:27.598131 27404 ts_tablet_manager.cc:1403] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.089s user 0.017s sys 0.033s
I20260807 01:46:27.601791 27484 consensus_queue.cc:1048] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:27.611088 27404 raft_consensus.cc:359] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.611991 27404 raft_consensus.cc:385] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:27.612329 27404 raft_consensus.cc:740] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:27.613200 27404 consensus_queue.cc:260] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.614928 27404 ts_tablet_manager.cc:1434] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.016s user 0.005s sys 0.002s
I20260807 01:46:27.615890 27404 tablet_bootstrap.cc:492] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:27.619037 27407 tablet_bootstrap.cc:654] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:27.623692 27461 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.661013 27405 tablet_bootstrap.cc:492] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:27.662999 27405 ts_tablet_manager.cc:1403] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.124s user 0.011s sys 0.033s
I20260807 01:46:27.663286 27404 tablet_bootstrap.cc:654] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:27.668695 27405 raft_consensus.cc:359] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.671463 27405 raft_consensus.cc:385] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:27.671849 27405 raft_consensus.cc:740] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:27.672658 27405 consensus_queue.cc:260] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.674504 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:27.675674 27405 ts_tablet_manager.cc:1434] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.011s user 0.003s sys 0.003s
I20260807 01:46:27.685061 27484 consensus_queue.cc:1048] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
W20260807 01:46:27.683357 27064 leader_election.cc:343] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:27.691241 27407 tablet_bootstrap.cc:492] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:27.699295 27407 ts_tablet_manager.cc:1403] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.145s user 0.009s sys 0.021s
I20260807 01:46:27.704499 27407 raft_consensus.cc:359] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.706400 27407 raft_consensus.cc:385] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:27.706808 27407 raft_consensus.cc:740] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:27.707842 27407 consensus_queue.cc:260] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.710328 27407 ts_tablet_manager.cc:1434] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.011s user 0.008s sys 0.001s
I20260807 01:46:27.710320 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:27.711427 27407 tablet_bootstrap.cc:492] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:27.711606 27333 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:27.711918 27461 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:27.696455 27405 tablet_bootstrap.cc:492] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:27.689486 27404 tablet_bootstrap.cc:492] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:27.676079 27495 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:27.714840 27495 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.716216 27461 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:27.717083 27461 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.722563 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:27.718058 27404 ts_tablet_manager.cc:1403] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.102s user 0.008s sys 0.008s
I20260807 01:46:27.723677 27229 raft_consensus.cc:2468] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
I20260807 01:46:27.725289 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:27.728087 27064 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:27.730345 27629 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:27.730935 27572 raft_consensus.cc:2804] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:27.731530 27572 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:27.731146 27629 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.732188 27572 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
W20260807 01:46:27.729830 27064 leader_election.cc:343] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
W20260807 01:46:27.733794 27036 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 01:46:27.735015 27494 raft_consensus.cc:493] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:27.746956 27494 raft_consensus.cc:515] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.749747 27404 raft_consensus.cc:359] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.750224 27494 leader_election.cc:290] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:27.750669 27495 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:27.751652 27629 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:27.750550 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:27.754112 27404 raft_consensus.cc:385] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:27.754451 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:27.755254 27337 raft_consensus.cc:2468] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:27.754475 27404 raft_consensus.cc:740] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:27.754492 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:27.757404 27404 consensus_queue.cc:260] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:27.756404 27287 leader_election.cc:343] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
W20260807 01:46:27.759232 27287 leader_election.cc:343] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:27.759649 27287 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:27.761240 27414 raft_consensus.cc:2749] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:27.766314 27170 leader_election.cc:304] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:27.767091 27404 ts_tablet_manager.cc:1434] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.043s user 0.005s sys 0.003s
I20260807 01:46:27.767827 27453 raft_consensus.cc:2804] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:27.768020 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:27.769122 27108 raft_consensus.cc:2468] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:27.768298 27453 raft_consensus.cc:493] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:27.770404 27453 raft_consensus.cc:3060] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:27.782367 27461 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:27.789067 27404 tablet_bootstrap.cc:492] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:27.787431 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:27.792644 27333 raft_consensus.cc:2468] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:27.793514 27337 raft_consensus.cc:2468] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
I20260807 01:46:27.792896 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:27.798100 27453 raft_consensus.cc:515] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.804283 27633 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:27.805183 27453 leader_election.cc:290] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:27.805086 27633 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.806598 27063 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:27.807369 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:27.810590 27108 raft_consensus.cc:3060] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:27.811372 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:27.809013 27633 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:27.814580 27104 raft_consensus.cc:2468] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:27.823201 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:27.823760 27287 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:27.824857 27484 raft_consensus.cc:2804] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260807 01:46:27.825145 27287 leader_election.cc:343] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:27.825860 27484 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:27.815269 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:27.826399 27484 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:27.826615 27336 raft_consensus.cc:3060] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
W20260807 01:46:27.829319 27063 leader_election.cc:343] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:27.831218 27063 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:27.832378 27495 raft_consensus.cc:2749] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:27.828287 27461 raft_consensus.cc:2804] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:27.837105 27461 raft_consensus.cc:697] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:27.839309 27405 tablet_bootstrap.cc:654] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:27.839046 27461 consensus_queue.cc:237] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.840699 27572 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.845956 27484 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.850375 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:27.854671 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:27.855165 27226 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:27.852702 27484 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:27.854671 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:27.856849 27407 tablet_bootstrap.cc:654] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:27.858654 27572 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:46:27.859066 27287 leader_election.cc:343] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:27.857242 27110 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:27.869234 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:27.869951 27333 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:27.870576 27404 tablet_bootstrap.cc:654] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:27.866020 27336 raft_consensus.cc:2468] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:27.874564 27108 raft_consensus.cc:2468] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:27.878600 27170 leader_election.cc:304] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:27.879817 27494 raft_consensus.cc:2804] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:27.880896 27494 raft_consensus.cc:697] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:27.884093 27494 consensus_queue.cc:237] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.887691 27633 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:27.888607 27633 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.896277 27633 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:27.901867 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:27.903106 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:27.902876 27405 tablet_bootstrap.cc:492] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
W20260807 01:46:27.905321 27287 leader_election.cc:343] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
W20260807 01:46:27.906337 27287 leader_election.cc:343] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:27.906769 27287 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:27.905769 27405 ts_tablet_manager.cc:1403] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.209s user 0.025s sys 0.003s
I20260807 01:46:27.907652 27633 raft_consensus.cc:2749] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:27.916069 27405 raft_consensus.cc:359] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.916852 27405 raft_consensus.cc:385] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:27.917138 27405 raft_consensus.cc:740] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:27.919209 27405 consensus_queue.cc:260] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.914839 27226 raft_consensus.cc:2468] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:27.923704 27404 tablet_bootstrap.cc:492] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:27.925028 27404 ts_tablet_manager.cc:1403] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.136s user 0.019s sys 0.032s
I20260807 01:46:27.926512 27064 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:27.927874 27572 raft_consensus.cc:2804] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:27.928647 27572 raft_consensus.cc:697] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:27.929494 27404 raft_consensus.cc:359] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.931819 27404 raft_consensus.cc:385] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:27.932159 27404 raft_consensus.cc:740] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:27.931561 27572 consensus_queue.cc:237] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.932962 27404 consensus_queue.cc:260] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:27.939740 27404 ts_tablet_manager.cc:1434] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.014s user 0.004s sys 0.004s
I20260807 01:46:27.972922 27405 ts_tablet_manager.cc:1434] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.065s user 0.008s sys 0.006s
I20260807 01:46:27.981379 27405 tablet_bootstrap.cc:492] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:28.004797 27110 raft_consensus.cc:2468] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:28.048025 27404 tablet_bootstrap.cc:492] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:28.053951 27333 raft_consensus.cc:2468] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:28.065793 27287 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:28.066324 26952 catalog_manager.cc:5697] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:28.070782 27633 raft_consensus.cc:2804] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:28.074326 27407 tablet_bootstrap.cc:492] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:28.074249 27633 raft_consensus.cc:697] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:28.074945 27407 ts_tablet_manager.cc:1403] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.364s user 0.019s sys 0.073s
I20260807 01:46:28.075479 27633 consensus_queue.cc:237] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.080417 27407 raft_consensus.cc:359] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.080989 27407 raft_consensus.cc:385] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:28.081437 26950 catalog_manager.cc:5697] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:28.081230 27407 raft_consensus.cc:740] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:28.092171 27407 consensus_queue.cc:260] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.094307 27404 tablet_bootstrap.cc:654] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:28.093858 27407 ts_tablet_manager.cc:1434] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.019s user 0.004s sys 0.000s
I20260807 01:46:28.099156 27405 tablet_bootstrap.cc:654] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:28.112905 27407 tablet_bootstrap.cc:492] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:28.112344 26954 catalog_manager.cc:5697] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:28.156896 27405 tablet_bootstrap.cc:492] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:28.157409 27405 ts_tablet_manager.cc:1403] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.176s user 0.030s sys 0.025s
I20260807 01:46:28.158268 27404 tablet_bootstrap.cc:492] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:28.158787 27404 ts_tablet_manager.cc:1403] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.218s user 0.015s sys 0.004s
I20260807 01:46:28.158702 27407 tablet_bootstrap.cc:654] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:28.162961 27404 raft_consensus.cc:359] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.163882 27404 raft_consensus.cc:385] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:28.164163 27404 raft_consensus.cc:740] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:28.162935 27405 raft_consensus.cc:359] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.164979 27405 raft_consensus.cc:385] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:28.165268 27405 raft_consensus.cc:740] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:28.166008 27405 consensus_queue.cc:260] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.166640 27404 consensus_queue.cc:260] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.182050 27405 ts_tablet_manager.cc:1434] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.024s user 0.015s sys 0.007s
I20260807 01:46:28.187485 27404 ts_tablet_manager.cc:1434] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.028s user 0.007s sys 0.001s
I20260807 01:46:28.196058 27702 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:28.196947 27702 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.199967 27405 tablet_bootstrap.cc:492] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:28.202068 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:28.203380 27333 raft_consensus.cc:2393] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 1: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:28.205137 27404 tablet_bootstrap.cc:492] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:28.200194 27702 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:28.219486 26950 catalog_manager.cc:5697] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:28.223230 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:28.230665 27407 tablet_bootstrap.cc:492] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:28.231354 27407 ts_tablet_manager.cc:1403] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.133s user 0.033s sys 0.020s
I20260807 01:46:28.233283 27172 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:28.235349 27702 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:28.235414 27407 raft_consensus.cc:359] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.237155 27407 raft_consensus.cc:385] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:28.237528 27407 raft_consensus.cc:740] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:28.238495 27407 consensus_queue.cc:260] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.239779 27405 tablet_bootstrap.cc:654] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:28.242074 27407 ts_tablet_manager.cc:1434] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.010s user 0.004s sys 0.002s
I20260807 01:46:28.243471 27407 tablet_bootstrap.cc:492] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:28.250461 27702 raft_consensus.cc:2749] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:28.266462 27404 tablet_bootstrap.cc:654] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:28.266523 27407 tablet_bootstrap.cc:654] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:28.273229 27414 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:28.274107 27414 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.276013 27405 tablet_bootstrap.cc:492] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:28.277096 27405 ts_tablet_manager.cc:1403] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.077s user 0.012s sys 0.011s
I20260807 01:46:28.279938 27572 consensus_queue.cc:1048] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:28.280395 27629 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:28.281112 27629 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.285177 27629 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:28.288084 27414 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:28.289439 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:28.290299 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:28.291440 27110 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:28.292927 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:28.294831 27405 raft_consensus.cc:359] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.295595 27405 raft_consensus.cc:385] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:28.295498 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:28.296514 27405 raft_consensus.cc:740] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
W20260807 01:46:28.296947 27287 leader_election.cc:343] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:28.296882 27407 tablet_bootstrap.cc:492] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:28.297737 27407 ts_tablet_manager.cc:1403] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.054s user 0.011s sys 0.014s
I20260807 01:46:28.297408 27405 consensus_queue.cc:260] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.300002 27405 ts_tablet_manager.cc:1434] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.023s user 0.006s sys 0.002s
I20260807 01:46:28.301447 27287 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
W20260807 01:46:28.302584 27287 leader_election.cc:343] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:28.302503 27414 raft_consensus.cc:2804] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:28.303285 27414 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:28.304005 27414 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:28.305195 27404 tablet_bootstrap.cc:492] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:28.305204 27407 raft_consensus.cc:359] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.306146 27407 raft_consensus.cc:385] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260807 01:46:28.306218 27287 leader_election.cc:343] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:28.306520 27407 raft_consensus.cc:740] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:28.307132 27404 ts_tablet_manager.cc:1403] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.116s user 0.005s sys 0.019s
I20260807 01:46:28.307488 27287 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:28.307387 27407 consensus_queue.cc:260] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.313277 27407 ts_tablet_manager.cc:1434] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.015s user 0.003s sys 0.006s
I20260807 01:46:28.314429 27407 tablet_bootstrap.cc:492] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:28.318104 27484 raft_consensus.cc:2749] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:28.323673 27404 raft_consensus.cc:359] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.324445 27404 raft_consensus.cc:385] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:28.324745 27404 raft_consensus.cc:740] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:28.328084 27405 tablet_bootstrap.cc:492] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:28.332695 27572 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:28.334084 27407 tablet_bootstrap.cc:654] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:28.325429 27404 consensus_queue.cc:260] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.339416 27414 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.353427 27572 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.355149 27404 ts_tablet_manager.cc:1434] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.046s user 0.005s sys 0.002s
I20260807 01:46:28.356930 27404 tablet_bootstrap.cc:492] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:28.356859 27414 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:28.358212 27572 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:28.358548 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:28.359282 27108 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:28.360450 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:28.363246 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:46:28.364713 27287 leader_election.cc:343] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:28.364199 27336 raft_consensus.cc:2468] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
I20260807 01:46:28.368445 27063 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:28.369475 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:28.369971 27461 consensus_queue.cc:1048] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:28.372615 27461 raft_consensus.cc:2804] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:28.373199 27461 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:28.373701 27461 raft_consensus.cc:3060] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:28.396512 27572 raft_consensus.cc:493] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:46:28.399603 27064 leader_election.cc:343] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:28.400082 27572 raft_consensus.cc:515] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.408963 27404 tablet_bootstrap.cc:654] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:28.415196 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:28.416404 27226 raft_consensus.cc:2468] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
I20260807 01:46:28.413944 27572 leader_election.cc:290] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:28.419983 27064 leader_election.cc:304] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:28.423405 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:28.422811 27495 consensus_queue.cc:1048] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260807 01:46:28.425763 27063 leader_election.cc:343] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:28.427778 27405 tablet_bootstrap.cc:654] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:28.431221 27407 tablet_bootstrap.cc:492] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:28.432338 27572 raft_consensus.cc:2804] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:28.435914 27407 ts_tablet_manager.cc:1403] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.122s user 0.019s sys 0.019s
I20260807 01:46:28.436347 27572 raft_consensus.cc:493] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:28.439512 27572 raft_consensus.cc:3060] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:28.442143 27108 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:28.450640 27287 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:28.452638 27414 raft_consensus.cc:2804] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:28.447880 27724 consensus_queue.cc:1048] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260807 01:46:28.456400 27414 raft_consensus.cc:697] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:28.463999 27495 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:28.466492 27495 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.467002 27414 consensus_queue.cc:237] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.474277 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:28.477823 27495 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:28.482528 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:46:28.486801 27063 leader_election.cc:343] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:28.497205 27407 raft_consensus.cc:359] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.497961 27407 raft_consensus.cc:385] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:28.498271 27407 raft_consensus.cc:740] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:28.498857 27407 consensus_queue.cc:260] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.502110 27404 tablet_bootstrap.cc:492] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:28.502468 27405 tablet_bootstrap.cc:492] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:28.502722 27404 ts_tablet_manager.cc:1403] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.147s user 0.022s sys 0.016s
I20260807 01:46:28.502956 27405 ts_tablet_manager.cc:1403] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.175s user 0.018s sys 0.008s
I20260807 01:46:28.509320 27404 raft_consensus.cc:359] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.510142 27404 raft_consensus.cc:385] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:28.514138 27404 raft_consensus.cc:740] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:28.514883 27404 consensus_queue.cc:260] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.519560 27407 ts_tablet_manager.cc:1434] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.080s user 0.005s sys 0.002s
I20260807 01:46:28.520099 27404 ts_tablet_manager.cc:1434] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.017s user 0.008s sys 0.000s
I20260807 01:46:28.522934 27404 tablet_bootstrap.cc:492] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:28.531924 27405 raft_consensus.cc:359] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.532660 27405 raft_consensus.cc:385] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:28.532944 27405 raft_consensus.cc:740] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:28.533293 27702 consensus_queue.cc:1048] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:28.533712 27405 consensus_queue.cc:260] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.542210 27407 tablet_bootstrap.cc:492] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:28.548545 26950 catalog_manager.cc:5697] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:28.555591 27405 ts_tablet_manager.cc:1434] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.052s user 0.007s sys 0.005s
I20260807 01:46:28.550119 27702 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:28.560443 27461 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:28.552208 27064 leader_election.cc:343] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:28.568184 27405 tablet_bootstrap.cc:492] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:28.540315 27572 raft_consensus.cc:515] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.565207 27702 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.589329 27572 leader_election.cc:290] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:28.592706 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:28.574853 27064 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
W20260807 01:46:28.596143 27063 leader_election.cc:343] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:28.598120 27572 raft_consensus.cc:2749] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:28.603997 27702 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:28.610972 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:28.615296 27404 tablet_bootstrap.cc:654] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
W20260807 01:46:28.597649 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 45917us
I20260807 01:46:28.623734 27461 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:28.624979 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:28.629108 27333 raft_consensus.cc:3060] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:28.639436 27484 consensus_queue.cc:1048] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:28.640894 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:28.643960 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:28.644685 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:28.645817 27105 raft_consensus.cc:2393] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 1: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:28.623943 27702 consensus_queue.cc:1048] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260807 01:46:28.644789 27229 raft_consensus.cc:3060] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:28.652264 27172 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:28.655431 27450 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:28.648061 27407 tablet_bootstrap.cc:654] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:28.659078 27405 tablet_bootstrap.cc:654] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
W20260807 01:46:28.666617 27064 leader_election.cc:343] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:28.731777 27404 tablet_bootstrap.cc:492] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:28.732416 27404 ts_tablet_manager.cc:1403] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.210s user 0.030s sys 0.033s
I20260807 01:46:28.721093 27333 raft_consensus.cc:2468] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:28.755183 27063 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:28.757344 27572 raft_consensus.cc:2804] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:28.758729 27572 raft_consensus.cc:697] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:28.759487 27450 raft_consensus.cc:2749] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:28.760289 27229 raft_consensus.cc:2468] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:28.765275 27633 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:28.765992 27633 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.760216 27572 consensus_queue.cc:237] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.770398 27064 leader_election.cc:304] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:46:28.775771 27461 raft_consensus.cc:2804] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:28.776496 27461 raft_consensus.cc:697] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:28.777024 27633 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:28.785336 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:28.787595 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:28.794348 27287 leader_election.cc:343] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:28.777823 27461 consensus_queue.cc:237] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.792783 27633 consensus_queue.cc:1048] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
W20260807 01:46:28.823916 27287 leader_election.cc:343] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:28.824446 27287 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:28.830442 27404 raft_consensus.cc:359] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.831179 27405 tablet_bootstrap.cc:492] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:28.831153 27404 raft_consensus.cc:385] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:28.831732 27404 raft_consensus.cc:740] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:28.831895 27405 ts_tablet_manager.cc:1403] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.264s user 0.004s sys 0.031s
I20260807 01:46:28.832844 27629 raft_consensus.cc:2749] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:28.845938 27407 tablet_bootstrap.cc:492] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:28.846508 27407 ts_tablet_manager.cc:1403] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.305s user 0.028s sys 0.023s
I20260807 01:46:28.833436 27404 consensus_queue.cc:260] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.848641 27404 ts_tablet_manager.cc:1434] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.116s user 0.004s sys 0.004s
I20260807 01:46:28.849884 27404 tablet_bootstrap.cc:492] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:28.853677 27407 raft_consensus.cc:359] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.854516 27407 raft_consensus.cc:385] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:28.854807 27407 raft_consensus.cc:740] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:28.870961 27407 consensus_queue.cc:260] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.887265 26952 catalog_manager.cc:5697] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:28.889063 26952 catalog_manager.cc:5697] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:28.896925 27405 raft_consensus.cc:359] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.897706 27405 raft_consensus.cc:385] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:28.897980 27405 raft_consensus.cc:740] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:28.900935 27707 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:28.901165 27405 consensus_queue.cc:260] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.901674 27707 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.903162 27404 tablet_bootstrap.cc:654] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:28.906970 27407 ts_tablet_manager.cc:1434] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.060s user 0.008s sys 0.000s
I20260807 01:46:28.910950 27405 ts_tablet_manager.cc:1434] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.079s user 0.007s sys 0.000s
I20260807 01:46:28.905925 27572 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:28.918720 27407 tablet_bootstrap.cc:492] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:28.915074 27572 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.919708 27405 tablet_bootstrap.cc:492] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:28.932370 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:28.933558 27105 raft_consensus.cc:2393] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 1: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:28.938113 27707 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:28.945492 27758 raft_consensus.cc:493] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:28.946368 27758 raft_consensus.cc:515] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.958971 27758 leader_election.cc:290] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:28.963759 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:28.971230 27170 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:28.972014 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:28.975185 27229 raft_consensus.cc:2393] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 1: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:28.973974 27752 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:28.982177 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:28.983610 27404 tablet_bootstrap.cc:492] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:28.984133 27404 ts_tablet_manager.cc:1403] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.134s user 0.012s sys 0.027s
I20260807 01:46:28.985688 27572 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:28.986763 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:28.988054 27287 leader_election.cc:304] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:28.990288 27414 raft_consensus.cc:3060] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:28.994179 27405 tablet_bootstrap.cc:654] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:28.988982 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:28.994975 27333 raft_consensus.cc:2468] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
I20260807 01:46:28.988104 27404 raft_consensus.cc:359] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:28.996016 27404 raft_consensus.cc:385] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:28.996225 27404 raft_consensus.cc:740] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
W20260807 01:46:28.995353 27064 leader_election.cc:343] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:28.999229 27063 leader_election.cc:304] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:28.996768 27404 consensus_queue.cc:260] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.002588 27404 ts_tablet_manager.cc:1434] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.018s user 0.007s sys 0.003s
I20260807 01:46:29.002451 27633 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:29.001009 27572 raft_consensus.cc:2804] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:29.003264 27572 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:29.003533 27633 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.004221 27572 raft_consensus.cc:3060] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:29.007866 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:29.007905 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:29.008881 27104 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:29.006754 27633 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:29.010818 27287 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
W20260807 01:46:29.011781 27287 leader_election.cc:343] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:29.012295 27633 raft_consensus.cc:2804] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:29.012928 27633 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:29.013368 27633 raft_consensus.cc:3060] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:29.015424 27407 tablet_bootstrap.cc:654] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:29.005757 27414 raft_consensus.cc:2749] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:29.011552 27572 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.021337 27572 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:29.022210 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:29.023033 27333 raft_consensus.cc:3060] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:29.024679 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:29.028106 27404 tablet_bootstrap.cc:492] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:29.018707 27752 raft_consensus.cc:2749] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:29.031034 27633 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.033638 27333 raft_consensus.cc:2468] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
W20260807 01:46:29.017467 27036 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 01:46:29.034529 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:29.035686 27104 raft_consensus.cc:3060] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
W20260807 01:46:29.028313 27064 leader_election.cc:343] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:29.052464 27633 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:29.061822 27404 tablet_bootstrap.cc:654] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:29.062701 27414 consensus_queue.cc:1048] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:29.066689 27063 leader_election.cc:304] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:29.067551 27572 raft_consensus.cc:2804] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:29.068149 27572 raft_consensus.cc:697] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:29.068876 27405 tablet_bootstrap.cc:492] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:29.069386 27405 ts_tablet_manager.cc:1403] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.150s user 0.021s sys 0.039s
I20260807 01:46:29.069206 27572 consensus_queue.cc:237] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.073568 27407 tablet_bootstrap.cc:492] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:29.076797 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:29.074213 27407 ts_tablet_manager.cc:1403] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.156s user 0.019s sys 0.022s
I20260807 01:46:29.077535 27405 raft_consensus.cc:359] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.078751 27405 raft_consensus.cc:385] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:29.079069 27405 raft_consensus.cc:740] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
W20260807 01:46:29.081568 27287 leader_election.cc:343] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:29.082276 27633 consensus_queue.cc:1048] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:29.079809 27405 consensus_queue.cc:260] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.093087 27407 raft_consensus.cc:359] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.094014 27407 raft_consensus.cc:385] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:29.094710 27407 raft_consensus.cc:740] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:29.101509 27104 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:29.102005 27405 ts_tablet_manager.cc:1434] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.032s user 0.003s sys 0.007s
I20260807 01:46:29.102207 27404 tablet_bootstrap.cc:492] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:29.102669 27404 ts_tablet_manager.cc:1403] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.075s user 0.018s sys 0.017s
I20260807 01:46:29.103348 27405 tablet_bootstrap.cc:492] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:29.106066 27404 raft_consensus.cc:359] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.095530 27407 consensus_queue.cc:260] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.106782 27404 raft_consensus.cc:385] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:29.107069 27404 raft_consensus.cc:740] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:29.107812 27404 consensus_queue.cc:260] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.108630 27407 ts_tablet_manager.cc:1434] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.030s user 0.007s sys 0.001s
I20260807 01:46:29.109691 27407 tablet_bootstrap.cc:492] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:29.110328 27404 ts_tablet_manager.cc:1434] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 01:46:29.111701 27287 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:29.112637 27629 raft_consensus.cc:2804] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:29.113446 27629 raft_consensus.cc:697] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:29.123461 27404 tablet_bootstrap.cc:492] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:29.137637 27629 consensus_queue.cc:237] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.168465 27407 tablet_bootstrap.cc:654] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:29.174098 27633 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:29.178973 27633 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.184118 27633 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:29.189592 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:29.190781 27104 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:29.193900 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:29.196084 27287 leader_election.cc:343] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:29.196846 27287 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:29.196235 26950 catalog_manager.cc:5697] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:29.197522 27758 raft_consensus.cc:2804] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:29.198071 27758 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:29.198625 27758 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:29.201126 27405 tablet_bootstrap.cc:654] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:29.264716 27407 tablet_bootstrap.cc:492] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:29.265367 27407 ts_tablet_manager.cc:1403] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.156s user 0.026s sys 0.013s
I20260807 01:46:29.270011 27404 tablet_bootstrap.cc:654] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
W20260807 01:46:29.282949 27064 consensus_peers.cc:597] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 -> Peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Couldn't send request to peer 3478729505b046c2a0051c2d0130873f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:29.274763 27758 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.291597 27407 raft_consensus.cc:359] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.292402 27407 raft_consensus.cc:385] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:29.292699 27407 raft_consensus.cc:740] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:29.293522 27407 consensus_queue.cc:260] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.302122 27407 ts_tablet_manager.cc:1434] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.036s user 0.008s sys 0.001s
I20260807 01:46:29.304112 27758 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:29.305795 27407 tablet_bootstrap.cc:492] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:29.307274 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:29.308279 27110 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:29.311048 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
W20260807 01:46:29.313053 27287 leader_election.cc:343] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:29.312734 26952 catalog_manager.cc:5697] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:29.317413 27405 tablet_bootstrap.cc:492] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:29.317988 27405 ts_tablet_manager.cc:1403] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.215s user 0.014s sys 0.024s
I20260807 01:46:29.319175 27404 tablet_bootstrap.cc:492] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:29.321374 27404 ts_tablet_manager.cc:1403] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.198s user 0.015s sys 0.022s
I20260807 01:46:29.324687 27110 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:29.326227 27287 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:29.327033 27758 raft_consensus.cc:2804] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:29.327603 27758 raft_consensus.cc:697] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:29.327443 27404 raft_consensus.cc:359] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.328220 27404 raft_consensus.cc:385] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:29.330132 27404 raft_consensus.cc:740] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:29.328697 27758 consensus_queue.cc:237] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.337339 27405 raft_consensus.cc:359] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.338198 27405 raft_consensus.cc:385] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:29.339105 27405 raft_consensus.cc:740] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:29.339880 27405 consensus_queue.cc:260] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.342895 27405 ts_tablet_manager.cc:1434] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.025s user 0.006s sys 0.001s
I20260807 01:46:29.389680 27404 consensus_queue.cc:260] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.391974 27572 consensus_queue.cc:1048] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:29.396397 27407 tablet_bootstrap.cc:654] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:29.397684 27404 ts_tablet_manager.cc:1434] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.076s user 0.006s sys 0.002s
I20260807 01:46:29.389367 27405 tablet_bootstrap.cc:492] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:29.399072 27404 tablet_bootstrap.cc:492] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:29.407361 27752 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:29.408016 27752 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.408895 27707 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:29.409762 27707 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.419587 27752 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:29.421104 27707 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:29.423169 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:29.422988 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:29.424024 27337 raft_consensus.cc:2468] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:29.426434 27170 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:29.427603 27702 raft_consensus.cc:2804] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:29.428138 27702 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:29.428574 27702 raft_consensus.cc:3060] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:29.429484 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:29.432698 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:29.443075 27495 consensus_queue.cc:1048] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:29.442981 26950 catalog_manager.cc:5697] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
W20260807 01:46:29.436352 27172 leader_election.cc:343] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:29.445904 27741 consensus_queue.cc:1048] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:29.448076 27172 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:29.452159 27450 raft_consensus.cc:2749] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:29.453601 27404 tablet_bootstrap.cc:654] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:29.463021 27407 tablet_bootstrap.cc:492] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:29.463630 27407 ts_tablet_manager.cc:1403] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.158s user 0.010s sys 0.030s
I20260807 01:46:29.469552 27405 tablet_bootstrap.cc:654] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:29.476075 27407 raft_consensus.cc:359] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.476925 27407 raft_consensus.cc:385] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:29.477216 27407 raft_consensus.cc:740] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:29.478058 27407 consensus_queue.cc:260] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.481985 27404 tablet_bootstrap.cc:492] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:29.482586 27404 ts_tablet_manager.cc:1403] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.084s user 0.006s sys 0.019s
I20260807 01:46:29.483258 27407 ts_tablet_manager.cc:1434] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.019s user 0.007s sys 0.002s
I20260807 01:46:29.484359 27407 tablet_bootstrap.cc:492] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:29.484115 27702 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.489185 27702 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:29.490284 27633 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:29.490844 27633 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.491310 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:29.491977 27110 raft_consensus.cc:3060] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:29.492278 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:29.492875 27331 raft_consensus.cc:3060] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:29.492947 27633 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:29.500312 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:29.501056 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:29.501781 27104 raft_consensus.cc:2468] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
W20260807 01:46:29.502951 27287 leader_election.cc:343] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:29.503775 27287 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:29.505239 27407 tablet_bootstrap.cc:654] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:29.505823 27633 raft_consensus.cc:2804] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:29.507761 27633 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:29.510687 27633 raft_consensus.cc:3060] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:29.513252 27331 raft_consensus.cc:2468] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:29.517501 27404 raft_consensus.cc:359] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.518373 27404 raft_consensus.cc:385] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:29.518683 27404 raft_consensus.cc:740] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:29.519429 27404 consensus_queue.cc:260] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.520933 27170 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:29.521798 27404 ts_tablet_manager.cc:1434] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.039s user 0.003s sys 0.003s
I20260807 01:46:29.522622 27702 raft_consensus.cc:2804] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:29.524264 27702 raft_consensus.cc:697] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:29.525779 27702 consensus_queue.cc:237] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.545156 27572 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:46:29.548553 27064 consensus_peers.cc:597] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 -> Peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Couldn't send request to peer 3478729505b046c2a0051c2d0130873f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:29.550757 27404 tablet_bootstrap.cc:492] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:29.546355 27572 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.564013 27110 raft_consensus.cc:2468] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:29.569015 27405 tablet_bootstrap.cc:492] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:29.569763 27405 ts_tablet_manager.cc:1403] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.181s user 0.009s sys 0.012s
I20260807 01:46:29.547837 27752 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:29.584491 27752 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.592226 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:29.595082 27064 leader_election.cc:343] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:29.618242 27752 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:29.620673 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:29.621073 27405 raft_consensus.cc:359] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.621791 27405 raft_consensus.cc:385] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:29.622073 27405 raft_consensus.cc:740] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:29.622743 27405 consensus_queue.cc:260] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.632974 27407 tablet_bootstrap.cc:492] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:29.633569 27407 ts_tablet_manager.cc:1403] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.149s user 0.011s sys 0.061s
W20260807 01:46:29.639156 27172 leader_election.cc:343] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:29.637552 27407 raft_consensus.cc:359] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.637524 26952 catalog_manager.cc:5697] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:29.646678 27407 raft_consensus.cc:385] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260807 01:46:29.644071 27287 consensus_peers.cc:597] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade -> Peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Couldn't send request to peer 3478729505b046c2a0051c2d0130873f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:29.648144 27405 ts_tablet_manager.cc:1434] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.078s user 0.010s sys 0.000s
I20260807 01:46:29.647060 27407 raft_consensus.cc:740] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:29.657441 27405 tablet_bootstrap.cc:492] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:29.649019 27572 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:29.658052 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:29.659696 27336 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
W20260807 01:46:29.654273 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 50245.2us
I20260807 01:46:29.665364 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:29.671380 27404 tablet_bootstrap.cc:654] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:29.674511 27407 consensus_queue.cc:260] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.646135 27633 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.678306 27633 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:46:29.679646 27170 leader_election.cc:343] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:29.680570 27170 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:29.681361 27804 raft_consensus.cc:2749] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:29.684640 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:29.685140 27063 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:29.685626 27110 raft_consensus.cc:3060] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:29.687592 27495 raft_consensus.cc:2804] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:29.688594 27495 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:29.689126 27495 raft_consensus.cc:3060] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:29.692803 27407 ts_tablet_manager.cc:1434] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.059s user 0.003s sys 0.008s
I20260807 01:46:29.693988 27407 tablet_bootstrap.cc:492] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:29.696280 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:29.699182 27414 consensus_queue.cc:1048] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260807 01:46:29.703573 27287 leader_election.cc:343] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:29.723402 27405 tablet_bootstrap.cc:654] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:29.725275 27758 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:29.727669 27110 raft_consensus.cc:2468] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:29.728813 27287 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:29.728606 27495 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.725960 27758 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.730985 27414 raft_consensus.cc:2804] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:29.731645 27414 raft_consensus.cc:697] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:29.732173 27758 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:29.740752 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:29.741015 27414 consensus_queue.cc:237] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:29.744379 27064 leader_election.cc:343] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:29.749482 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:29.751376 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:29.747928 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 62888.4us
I20260807 01:46:29.761149 27495 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:46:29.762825 27287 leader_election.cc:343] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
W20260807 01:46:29.767310 27287 leader_election.cc:343] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:29.767701 27287 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:29.768918 27633 raft_consensus.cc:2749] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:29.775488 27227 raft_consensus.cc:3060] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:29.782440 27404 tablet_bootstrap.cc:492] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:29.783026 27404 ts_tablet_manager.cc:1403] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.250s user 0.022s sys 0.011s
I20260807 01:46:29.785444 27572 consensus_queue.cc:1048] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:29.785712 27404 raft_consensus.cc:359] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.786684 27404 raft_consensus.cc:385] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:29.786962 27404 raft_consensus.cc:740] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:29.787611 27404 consensus_queue.cc:260] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.792274 27404 ts_tablet_manager.cc:1434] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260807 01:46:29.796861 27404 tablet_bootstrap.cc:492] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:29.800479 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:29.802079 27336 raft_consensus.cc:3060] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:29.821910 27407 tablet_bootstrap.cc:654] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:29.832491 27405 tablet_bootstrap.cc:492] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:29.831943 26952 catalog_manager.cc:5697] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:29.833142 27405 ts_tablet_manager.cc:1403] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.178s user 0.017s sys 0.010s
I20260807 01:46:29.840385 27405 raft_consensus.cc:359] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.841161 27405 raft_consensus.cc:385] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:29.841434 27405 raft_consensus.cc:740] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:29.843552 27336 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:29.842073 27405 consensus_queue.cc:260] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.849797 27405 ts_tablet_manager.cc:1434] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.016s user 0.008s sys 0.000s
I20260807 01:46:29.850792 27404 tablet_bootstrap.cc:654] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:29.852571 27405 tablet_bootstrap.cc:492] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:29.846222 27063 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:29.867314 27741 raft_consensus.cc:2804] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:29.868187 27741 raft_consensus.cc:697] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:29.869927 27741 consensus_queue.cc:237] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.878264 27405 tablet_bootstrap.cc:654] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:29.881160 27407 tablet_bootstrap.cc:492] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:29.881704 27407 ts_tablet_manager.cc:1403] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.188s user 0.018s sys 0.022s
I20260807 01:46:29.884586 27407 raft_consensus.cc:359] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.885288 27633 raft_consensus.cc:493] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:29.886449 27633 raft_consensus.cc:515] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.887409 27407 raft_consensus.cc:385] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:29.887725 27407 raft_consensus.cc:740] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:29.892738 27633 leader_election.cc:290] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:29.893577 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:29.891343 27407 consensus_queue.cc:260] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:29.895008 27287 leader_election.cc:343] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:29.896847 27407 ts_tablet_manager.cc:1434] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.015s user 0.003s sys 0.003s
I20260807 01:46:29.896732 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:29.904608 27404 tablet_bootstrap.cc:492] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:29.906736 27404 ts_tablet_manager.cc:1403] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.110s user 0.016s sys 0.020s
I20260807 01:46:29.906219 27227 raft_consensus.cc:2468] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:29.907716 27407 tablet_bootstrap.cc:492] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:29.906760 27414 consensus_queue.cc:1048] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260807 01:46:29.913751 27287 leader_election.cc:304] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:29.916347 27404 raft_consensus.cc:359] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.920105 27404 raft_consensus.cc:385] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:29.920455 27404 raft_consensus.cc:740] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:29.921103 27405 tablet_bootstrap.cc:492] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:29.921682 27405 ts_tablet_manager.cc:1403] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.069s user 0.016s sys 0.002s
I20260807 01:46:29.921309 27404 consensus_queue.cc:260] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.933753 27633 raft_consensus.cc:2804] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:29.965215 27633 raft_consensus.cc:493] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:29.966089 27633 raft_consensus.cc:3060] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:29.968914 27404 ts_tablet_manager.cc:1434] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.062s user 0.005s sys 0.010s
I20260807 01:46:29.972350 27404 tablet_bootstrap.cc:492] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:29.995107 27808 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:29.976984 26952 catalog_manager.cc:5697] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:29.995697 27808 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.002224 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:30.027756 27407 tablet_bootstrap.cc:654] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:30.030623 27707 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:30.038841 27450 consensus_queue.cc:1048] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
W20260807 01:46:30.044860 27064 leader_election.cc:343] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:30.000941 27808 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:30.025700 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:30.043279 27707 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:29.995707 27629 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:30.050804 27629 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.051563 27707 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:30.054382 27784 raft_consensus.cc:493] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:30.055233 27784 raft_consensus.cc:515] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.056639 27752 consensus_queue.cc:1048] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:30.057001 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:46:30.058902 27287 consensus_peers.cc:597] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade -> Peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Couldn't send request to peer 3478729505b046c2a0051c2d0130873f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:30.060060 27405 raft_consensus.cc:359] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.062086 27405 raft_consensus.cc:385] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:30.062441 27405 raft_consensus.cc:740] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
W20260807 01:46:30.059880 27063 leader_election.cc:343] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:30.063570 27063 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:30.065531 27572 raft_consensus.cc:2749] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:30.065481 27405 consensus_queue.cc:260] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.069622 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:30.065327 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:30.058880 27784 leader_election.cc:290] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:30.081681 27404 tablet_bootstrap.cc:654] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:30.083480 27405 ts_tablet_manager.cc:1434] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.161s user 0.013s sys 0.013s
I20260807 01:46:30.054397 27629 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:30.016191 27633 raft_consensus.cc:515] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.090754 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:30.110528 27405 tablet_bootstrap.cc:492] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:30.086215 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:30.081830 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:46:30.112432 27287 leader_election.cc:343] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
W20260807 01:46:30.114661 27287 leader_election.cc:343] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:30.114986 27287 leader_election.cc:304] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:30.102624 27227 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:30.122568 27407 tablet_bootstrap.cc:492] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:30.123045 27407 ts_tablet_manager.cc:1403] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.216s user 0.021s sys 0.011s
W20260807 01:46:30.125188 27287 leader_election.cc:343] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:30.117430 27629 raft_consensus.cc:2749] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:30.131582 27287 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:30.133618 27414 raft_consensus.cc:2804] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:30.134323 27414 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:30.134908 27414 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:30.138988 27227 raft_consensus.cc:3060] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:30.149655 27572 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:30.151679 27572 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.155301 27572 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:30.159346 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:30.162025 27064 leader_election.cc:343] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:30.168946 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:30.171074 27407 raft_consensus.cc:359] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.178445 27407 raft_consensus.cc:385] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:30.178880 27407 raft_consensus.cc:740] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:30.179826 27407 consensus_queue.cc:260] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.180943 27170 leader_election.cc:304] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:30.177968 27405 tablet_bootstrap.cc:654] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:30.185590 27407 ts_tablet_manager.cc:1434] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.062s user 0.009s sys 0.000s
I20260807 01:46:30.186820 27407 tablet_bootstrap.cc:492] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:30.180994 27414 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.193212 27633 leader_election.cc:290] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:46:30.199908 27036 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 01:46:30.210515 27414 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:46:30.216322 27287 leader_election.cc:343] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:30.217790 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:30.218932 27226 raft_consensus.cc:3060] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:30.224202 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:30.225613 27333 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
I20260807 01:46:30.228610 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:30.234647 27063 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:30.237284 27808 raft_consensus.cc:2804] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:30.237721 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:30.237947 27808 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:30.238711 27222 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:30.238721 27808 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
W20260807 01:46:30.242998 27287 leader_election.cc:343] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:30.272004 27404 tablet_bootstrap.cc:492] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:30.272572 27404 ts_tablet_manager.cc:1403] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.301s user 0.039s sys 0.027s
I20260807 01:46:30.275853 27222 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:30.276592 27404 raft_consensus.cc:359] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.277287 27404 raft_consensus.cc:385] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:30.277585 27404 raft_consensus.cc:740] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:30.278627 27808 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.280998 27404 consensus_queue.cc:260] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.281417 27808 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:30.283800 27287 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:30.286056 27414 raft_consensus.cc:2804] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:30.286692 27404 ts_tablet_manager.cc:1434] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.014s user 0.007s sys 0.001s
I20260807 01:46:30.288246 27784 consensus_queue.cc:1048] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:30.288661 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:30.289484 27333 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:30.290567 27404 tablet_bootstrap.cc:492] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:30.298012 27226 raft_consensus.cc:2468] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
W20260807 01:46:30.302251 27287 consensus_peers.cc:597] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade -> Peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Couldn't send request to peer 3478729505b046c2a0051c2d0130873f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:30.287565 27414 raft_consensus.cc:697] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:30.324474 27287 leader_election.cc:304] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:30.328660 27405 tablet_bootstrap.cc:492] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:30.335541 27405 ts_tablet_manager.cc:1403] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.225s user 0.020s sys 0.005s
I20260807 01:46:30.333312 27407 tablet_bootstrap.cc:654] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:30.337618 27629 raft_consensus.cc:2804] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:30.338380 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:30.338938 27629 raft_consensus.cc:697] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:30.340006 27414 consensus_queue.cc:237] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.340333 27629 consensus_queue.cc:237] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:30.342449 27064 leader_election.cc:343] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:30.345032 27404 tablet_bootstrap.cc:654] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:30.351066 27405 raft_consensus.cc:359] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.351794 27405 raft_consensus.cc:385] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:30.352070 27405 raft_consensus.cc:740] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:30.352763 27405 consensus_queue.cc:260] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.354983 27405 ts_tablet_manager.cc:1434] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.019s user 0.004s sys 0.006s
I20260807 01:46:30.362282 27702 raft_consensus.cc:2749] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:30.364910 27226 raft_consensus.cc:3060] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:30.368639 27407 tablet_bootstrap.cc:492] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:30.369194 27407 ts_tablet_manager.cc:1403] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.183s user 0.009s sys 0.007s
I20260807 01:46:30.376538 27405 tablet_bootstrap.cc:492] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:30.377746 27741 consensus_queue.cc:1048] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260807 01:46:30.359433 27333 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:30.400441 27063 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:30.402376 27572 raft_consensus.cc:2804] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:30.402932 27572 raft_consensus.cc:697] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:30.403630 27572 consensus_queue.cc:237] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.431236 27405 tablet_bootstrap.cc:654] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:30.437180 27407 raft_consensus.cc:359] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.437917 27407 raft_consensus.cc:385] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:30.438206 27407 raft_consensus.cc:740] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:30.438843 27407 consensus_queue.cc:260] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.440389 27407 ts_tablet_manager.cc:1434] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.071s user 0.004s sys 0.002s
I20260807 01:46:30.441349 27407 tablet_bootstrap.cc:492] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:30.443766 27404 tablet_bootstrap.cc:492] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:30.444229 27404 ts_tablet_manager.cc:1403] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.154s user 0.026s sys 0.001s
I20260807 01:46:30.450868 27404 raft_consensus.cc:359] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.451565 27404 raft_consensus.cc:385] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:30.451848 27404 raft_consensus.cc:740] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:30.452768 27404 consensus_queue.cc:260] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.456245 27404 ts_tablet_manager.cc:1434] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.012s user 0.007s sys 0.001s
I20260807 01:46:30.461316 27404 tablet_bootstrap.cc:492] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:30.460440 27741 consensus_queue.cc:1048] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:30.460562 27407 tablet_bootstrap.cc:654] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:30.472518 26950 catalog_manager.cc:5697] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:30.480757 26954 catalog_manager.cc:5697] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:30.489346 27404 tablet_bootstrap.cc:654] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:30.508685 27633 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:30.509895 27405 tablet_bootstrap.cc:492] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:30.509598 27633 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.517189 27572 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:30.517784 27572 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.531265 27405 ts_tablet_manager.cc:1403] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.167s user 0.025s sys 0.009s
I20260807 01:46:30.519641 27633 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:30.532591 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:30.530797 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:30.523682 27572 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:30.535595 27405 raft_consensus.cc:359] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.535795 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:30.536329 27405 raft_consensus.cc:385] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:30.536559 27405 raft_consensus.cc:740] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
W20260807 01:46:30.537523 27064 leader_election.cc:343] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:30.537184 27405 consensus_queue.cc:260] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.539726 27405 ts_tablet_manager.cc:1434] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260807 01:46:30.540213 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:46:30.541432 27287 leader_election.cc:343] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
W20260807 01:46:30.542409 27063 leader_election.cc:343] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
W20260807 01:46:30.543110 27287 leader_election.cc:343] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:30.543004 27063 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:30.543601 27287 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:30.544343 27414 raft_consensus.cc:2749] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:30.545467 27808 raft_consensus.cc:2749] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:30.545671 27405 tablet_bootstrap.cc:492] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:30.557768 27407 tablet_bootstrap.cc:492] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:30.558308 27407 ts_tablet_manager.cc:1403] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.117s user 0.017s sys 0.006s
I20260807 01:46:30.561367 27407 raft_consensus.cc:359] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.562071 27407 raft_consensus.cc:385] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:30.562774 27407 raft_consensus.cc:740] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:30.572103 26954 catalog_manager.cc:5697] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:30.582984 27407 consensus_queue.cc:260] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.585976 27407 ts_tablet_manager.cc:1434] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20260807 01:46:30.586139 27405 tablet_bootstrap.cc:654] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:30.593938 27752 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:30.594799 27752 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.600358 27404 tablet_bootstrap.cc:492] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:30.600950 27404 ts_tablet_manager.cc:1403] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.140s user 0.022s sys 0.026s
I20260807 01:46:30.605564 27752 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:30.606544 27407 tablet_bootstrap.cc:492] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:30.606769 27404 raft_consensus.cc:359] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.607591 27404 raft_consensus.cc:385] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:30.607486 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:30.610278 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:30.612560 27170 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:30.613885 27804 raft_consensus.cc:2749] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:30.615993 27405 tablet_bootstrap.cc:492] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:30.616562 27405 ts_tablet_manager.cc:1403] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.071s user 0.010s sys 0.009s
I20260807 01:46:30.622118 27407 tablet_bootstrap.cc:654] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:30.613895 27404 raft_consensus.cc:740] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:30.623699 27405 raft_consensus.cc:359] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.624445 27405 raft_consensus.cc:385] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:30.624734 27405 raft_consensus.cc:740] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:30.625443 27405 consensus_queue.cc:260] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.627903 27405 ts_tablet_manager.cc:1434] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.011s user 0.006s sys 0.003s
I20260807 01:46:30.629323 27405 tablet_bootstrap.cc:492] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:30.628576 27404 consensus_queue.cc:260] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.638903 27404 ts_tablet_manager.cc:1434] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.038s user 0.009s sys 0.004s
I20260807 01:46:30.640468 27404 tablet_bootstrap.cc:492] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:30.641307 27405 tablet_bootstrap.cc:654] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:30.657680 27404 tablet_bootstrap.cc:654] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:30.664089 27407 tablet_bootstrap.cc:492] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:30.664744 27407 ts_tablet_manager.cc:1403] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.074s user 0.008s sys 0.018s
I20260807 01:46:30.668534 27405 tablet_bootstrap.cc:492] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:30.669086 27405 ts_tablet_manager.cc:1403] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.040s user 0.019s sys 0.008s
I20260807 01:46:30.670692 27407 raft_consensus.cc:359] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.672283 27407 raft_consensus.cc:385] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:30.672554 27407 raft_consensus.cc:740] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:30.673202 27407 consensus_queue.cc:260] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.673537 27405 raft_consensus.cc:359] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.674365 27405 raft_consensus.cc:385] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:30.674665 27405 raft_consensus.cc:740] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:30.675423 27405 consensus_queue.cc:260] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.677573 27405 ts_tablet_manager.cc:1434] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20260807 01:46:30.678952 27405 tablet_bootstrap.cc:492] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:30.679550 27808 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:30.680418 27808 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.685153 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:30.687290 27064 leader_election.cc:343] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:30.689636 27404 tablet_bootstrap.cc:492] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:30.690194 27404 ts_tablet_manager.cc:1403] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.050s user 0.011s sys 0.009s
I20260807 01:46:30.691811 27405 tablet_bootstrap.cc:654] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:30.693595 27752 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:30.693897 27808 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:30.694480 27752 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.696497 27407 ts_tablet_manager.cc:1434] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.031s user 0.003s sys 0.007s
I20260807 01:46:30.698102 27407 tablet_bootstrap.cc:492] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:30.711478 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:46:30.713781 27172 leader_election.cc:343] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:30.715153 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:30.715806 27333 raft_consensus.cc:2468] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
I20260807 01:46:30.718228 27404 raft_consensus.cc:359] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.718986 27404 raft_consensus.cc:385] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:30.719244 27404 raft_consensus.cc:740] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:30.719888 27404 consensus_queue.cc:260] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.722563 27404 ts_tablet_manager.cc:1434] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.032s user 0.009s sys 0.003s
I20260807 01:46:30.723927 27404 tablet_bootstrap.cc:492] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:30.721731 27063 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:30.727779 27752 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:30.730585 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:30.731688 27338 raft_consensus.cc:2468] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:30.734030 27170 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:30.735610 27572 raft_consensus.cc:2804] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:30.736058 27572 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:30.736385 27572 raft_consensus.cc:3060] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:30.735221 27707 raft_consensus.cc:2804] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:30.737254 27707 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:30.737856 27707 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:30.745827 27404 tablet_bootstrap.cc:654] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:30.761325 27804 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:30.762312 27804 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.745950 27572 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.766116 27572 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:30.768054 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:30.768560 27804 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:30.769810 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:30.771277 27404 tablet_bootstrap.cc:492] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:30.771917 27404 ts_tablet_manager.cc:1403] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.048s user 0.008s sys 0.017s
I20260807 01:46:30.772033 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:30.773263 27170 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
W20260807 01:46:30.773874 27064 leader_election.cc:343] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:30.781961 27752 raft_consensus.cc:2749] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:30.783550 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:30.784796 27338 raft_consensus.cc:3060] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:30.792547 27407 tablet_bootstrap.cc:654] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:30.798115 27404 raft_consensus.cc:359] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.799088 27404 raft_consensus.cc:385] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:30.799420 27404 raft_consensus.cc:740] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:30.799638 27405 tablet_bootstrap.cc:492] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:30.800184 27405 ts_tablet_manager.cc:1403] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.121s user 0.027s sys 0.026s
I20260807 01:46:30.801343 27404 consensus_queue.cc:260] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.803753 27404 ts_tablet_manager.cc:1434] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.032s user 0.003s sys 0.006s
I20260807 01:46:30.804428 27572 raft_consensus.cc:493] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:30.804893 27404 tablet_bootstrap.cc:492] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:30.804965 27572 raft_consensus.cc:515] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.808203 27572 leader_election.cc:290] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:30.808384 27227 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:30.810451 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:30.813678 27333 raft_consensus.cc:2468] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
I20260807 01:46:30.816150 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:30.832263 27063 leader_election.cc:304] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:30.833482 27808 raft_consensus.cc:2804] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:30.833986 27808 raft_consensus.cc:493] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:30.834263 27407 tablet_bootstrap.cc:492] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:30.834568 27808 raft_consensus.cc:3060] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
W20260807 01:46:30.834820 27064 leader_election.cc:343] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:30.834811 27407 ts_tablet_manager.cc:1403] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.137s user 0.008s sys 0.019s
I20260807 01:46:30.836221 27633 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:30.837071 27633 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.841208 27633 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:30.843139 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:30.841212 27407 raft_consensus.cc:359] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.846395 27407 raft_consensus.cc:385] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:30.846712 27407 raft_consensus.cc:740] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
W20260807 01:46:30.850683 27287 leader_election.cc:343] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:30.854344 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:30.857201 27105 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
W20260807 01:46:30.857890 27287 leader_election.cc:343] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:30.858390 27287 leader_election.cc:304] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:30.858084 27407 consensus_queue.cc:260] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.859148 27414 raft_consensus.cc:2749] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:30.863868 27707 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.865765 27405 raft_consensus.cc:359] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.866562 27405 raft_consensus.cc:385] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:30.868603 27338 raft_consensus.cc:2468] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:30.870883 27407 ts_tablet_manager.cc:1434] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.035s user 0.007s sys 0.005s
I20260807 01:46:30.871953 27407 tablet_bootstrap.cc:492] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:30.873740 27063 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:30.866858 27405 raft_consensus.cc:740] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:30.875293 27851 raft_consensus.cc:2804] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:30.876925 27851 raft_consensus.cc:697] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:30.878213 27404 tablet_bootstrap.cc:654] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:30.885238 27741 consensus_queue.cc:1048] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:30.875774 27405 consensus_queue.cc:260] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.878540 27851 consensus_queue.cc:237] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.889932 27405 ts_tablet_manager.cc:1434] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.089s user 0.009s sys 0.006s
I20260807 01:46:30.895283 27405 tablet_bootstrap.cc:492] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:30.908794 27108 raft_consensus.cc:3060] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:30.918088 27414 consensus_queue.cc:1048] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:30.897869 27808 raft_consensus.cc:515] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.934827 27414 raft_consensus.cc:493] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:30.938427 27633 consensus_queue.cc:1048] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:30.937108 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:30.942612 26954 catalog_manager.cc:5697] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:30.935299 27414 raft_consensus.cc:515] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:30.946388 27172 leader_election.cc:343] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:30.951253 27414 leader_election.cc:290] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:30.954607 27407 tablet_bootstrap.cc:654] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:30.945773 27752 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:30.960393 27572 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:30.960947 27572 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.945322 27707 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:30.959390 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:30.956622 27804 raft_consensus.cc:493] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:46:30.972533 27287 leader_election.cc:343] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:30.975349 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:30.975355 27405 tablet_bootstrap.cc:654] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:30.971572 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:30.990244 27572 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:30.962603 27752 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.978431 27804 raft_consensus.cc:515] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:30.992003 27338 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:30.996639 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:31.007817 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:31.012020 27230 raft_consensus.cc:3060] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
W20260807 01:46:31.016916 27064 consensus_peers.cc:597] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 -> Peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Couldn't send request to peer 3478729505b046c2a0051c2d0130873f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:31.019677 27879 consensus_queue.cc:1048] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
W20260807 01:46:31.022241 27287 leader_election.cc:343] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:31.029488 27287 leader_election.cc:304] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:31.026917 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:31.036018 27104 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:31.026310 27808 leader_election.cc:290] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:31.023474 27752 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:31.048256 27804 leader_election.cc:290] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:31.018745 27226 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:31.043740 27172 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:31.077328 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:31.080694 27064 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
W20260807 01:46:31.083545 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 61367.8us
I20260807 01:46:31.090915 27404 tablet_bootstrap.cc:492] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:31.091468 27404 ts_tablet_manager.cc:1403] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.287s user 0.021s sys 0.025s
I20260807 01:46:31.091409 27414 consensus_queue.cc:1048] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:31.090488 27105 raft_consensus.cc:2393] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 1: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:31.101310 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:31.102175 27333 raft_consensus.cc:3060] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:31.104192 27404 raft_consensus.cc:359] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.104970 27404 raft_consensus.cc:385] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:31.105257 27404 raft_consensus.cc:740] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:31.086920 27450 raft_consensus.cc:2804] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:31.087884 27338 raft_consensus.cc:2468] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:31.161633 27333 raft_consensus.cc:2468] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:31.088614 27633 raft_consensus.cc:2749] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:31.093839 27878 raft_consensus.cc:2804] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:31.151688 27407 tablet_bootstrap.cc:492] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:31.105908 27404 consensus_queue.cc:260] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.164951 27407 ts_tablet_manager.cc:1403] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.293s user 0.026s sys 0.013s
I20260807 01:46:31.164083 27450 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:31.176584 27891 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:31.177603 27405 tablet_bootstrap.cc:492] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:31.178719 27405 ts_tablet_manager.cc:1403] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.284s user 0.008s sys 0.015s
I20260807 01:46:31.181732 27404 ts_tablet_manager.cc:1434] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.090s user 0.009s sys 0.001s
I20260807 01:46:31.163800 27878 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:31.169124 27450 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:31.197088 27405 raft_consensus.cc:359] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.197870 27405 raft_consensus.cc:385] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:31.200182 27405 raft_consensus.cc:740] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:31.200773 27405 consensus_queue.cc:260] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.190665 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:31.208004 27407 raft_consensus.cc:359] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.188156 27878 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:31.208830 27407 raft_consensus.cc:385] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:31.209620 27407 raft_consensus.cc:740] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:31.211650 27405 ts_tablet_manager.cc:1434] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.033s user 0.004s sys 0.004s
I20260807 01:46:31.213605 27405 tablet_bootstrap.cc:492] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:31.177583 27891 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.220265 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:31.221360 27328 raft_consensus.cc:2393] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 1: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:31.222883 27702 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:31.231343 27170 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:31.224280 27230 raft_consensus.cc:2468] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:31.233511 27804 raft_consensus.cc:2804] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:31.223419 27702 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.236126 27229 raft_consensus.cc:3060] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:31.247269 27702 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:31.236404 27804 raft_consensus.cc:697] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:31.252254 27804 consensus_queue.cc:237] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.255812 27405 tablet_bootstrap.cc:654] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:31.266229 27450 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.267505 27891 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:31.262338 27878 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.270505 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:31.271989 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:31.273808 27878 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:31.274362 27450 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:46:31.277002 27287 leader_election.cc:343] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:31.279099 27405 tablet_bootstrap.cc:492] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:31.279693 27405 ts_tablet_manager.cc:1403] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.066s user 0.015s sys 0.005s
I20260807 01:46:31.280534 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
W20260807 01:46:31.282728 27170 leader_election.cc:343] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:31.282697 27405 raft_consensus.cc:359] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.283478 27405 raft_consensus.cc:385] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:31.283782 27405 raft_consensus.cc:740] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:31.284380 27405 consensus_queue.cc:260] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.285724 27405 ts_tablet_manager.cc:1434] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260807 01:46:31.286556 27405 tablet_bootstrap.cc:492] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:31.294766 27405 tablet_bootstrap.cc:654] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:31.302729 27405 tablet_bootstrap.cc:492] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:31.303180 27405 ts_tablet_manager.cc:1403] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.005s
I20260807 01:46:31.306617 27405 raft_consensus.cc:359] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.307199 27405 raft_consensus.cc:385] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:31.307377 27405 raft_consensus.cc:740] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:31.308051 27405 consensus_queue.cc:260] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.309800 27405 ts_tablet_manager.cc:1434] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.006s user 0.002s sys 0.002s
I20260807 01:46:31.310945 27405 tablet_bootstrap.cc:492] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:31.320833 27405 tablet_bootstrap.cc:654] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:31.321519 27450 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:31.322088 27450 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.324285 27450 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:31.325276 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:31.325888 27330 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:31.326889 27170 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:31.327491 27450 raft_consensus.cc:2804] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:31.327775 27450 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:31.328016 27450 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:31.329807 27405 tablet_bootstrap.cc:492] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:31.330313 27405 ts_tablet_manager.cc:1403] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.020s user 0.014s sys 0.005s
I20260807 01:46:31.332767 27450 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.334407 27450 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:31.334249 27405 raft_consensus.cc:359] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.335072 27405 raft_consensus.cc:385] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:31.335378 27405 raft_consensus.cc:740] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:31.336143 27405 consensus_queue.cc:260] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.336380 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:31.337504 27330 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:31.337896 27405 ts_tablet_manager.cc:1434] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260807 01:46:31.339052 27405 tablet_bootstrap.cc:492] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:31.351035 27405 tablet_bootstrap.cc:654] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:31.353111 27330 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:31.355058 27170 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:31.356098 27450 raft_consensus.cc:2804] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:31.356616 27450 raft_consensus.cc:697] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:31.357388 27450 consensus_queue.cc:237] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.360900 27405 tablet_bootstrap.cc:492] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:31.361330 27405 ts_tablet_manager.cc:1403] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.022s user 0.011s sys 0.009s
I20260807 01:46:31.365293 27405 raft_consensus.cc:359] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.365865 27405 raft_consensus.cc:385] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:31.366042 27405 raft_consensus.cc:740] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:31.366650 27405 consensus_queue.cc:260] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.368561 27405 ts_tablet_manager.cc:1434] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260807 01:46:31.371191 27405 tablet_bootstrap.cc:492] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:31.378139 27229 raft_consensus.cc:3060] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:31.382802 27450 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:31.383826 27405 tablet_bootstrap.cc:654] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:31.383441 27450 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.387384 27450 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:31.387957 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:31.393610 27405 tablet_bootstrap.cc:492] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:31.394083 27405 ts_tablet_manager.cc:1403] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.023s user 0.008s sys 0.012s
I20260807 01:46:31.397270 27405 raft_consensus.cc:359] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.397794 27405 raft_consensus.cc:385] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:31.397996 27405 raft_consensus.cc:740] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:31.398651 27405 consensus_queue.cc:260] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.400081 27405 ts_tablet_manager.cc:1434] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260807 01:46:31.401438 27405 tablet_bootstrap.cc:492] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:31.405391 27450 raft_consensus.cc:493] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:31.405936 27450 raft_consensus.cc:515] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.408092 27450 leader_election.cc:290] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:31.408830 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:31.409555 27330 raft_consensus.cc:2468] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:31.410668 27170 leader_election.cc:304] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:31.411444 27450 raft_consensus.cc:2804] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:31.411841 27450 raft_consensus.cc:493] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:31.412144 27405 tablet_bootstrap.cc:654] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:31.412173 27450 raft_consensus.cc:3060] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:31.419317 27450 raft_consensus.cc:515] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.420037 27405 tablet_bootstrap.cc:492] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:31.420535 27405 ts_tablet_manager.cc:1403] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.016s
I20260807 01:46:31.421774 27450 leader_election.cc:290] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:31.422524 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:31.423219 27330 raft_consensus.cc:3060] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:31.424113 27405 raft_consensus.cc:359] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.424898 27405 raft_consensus.cc:385] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:31.425112 27405 raft_consensus.cc:740] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:31.425736 27405 consensus_queue.cc:260] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.427191 27405 ts_tablet_manager.cc:1434] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.006s user 0.004s sys 0.004s
I20260807 01:46:31.428036 27405 tablet_bootstrap.cc:492] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:31.430912 27330 raft_consensus.cc:2468] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:31.432631 27170 leader_election.cc:304] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:31.433316 27804 raft_consensus.cc:2804] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:31.433756 27804 raft_consensus.cc:697] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:31.434754 27804 consensus_queue.cc:237] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.436165 27405 tablet_bootstrap.cc:654] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:31.457278 27405 tablet_bootstrap.cc:492] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:31.458140 27405 ts_tablet_manager.cc:1403] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.020s
I20260807 01:46:31.460798 27405 raft_consensus.cc:359] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.461301 27405 raft_consensus.cc:385] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:31.461470 27405 raft_consensus.cc:740] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:31.462023 27405 consensus_queue.cc:260] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.463313 27405 ts_tablet_manager.cc:1434] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260807 01:46:31.464032 27405 tablet_bootstrap.cc:492] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:31.470620 27405 tablet_bootstrap.cc:654] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:31.477020 27405 tablet_bootstrap.cc:492] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:31.477459 27405 ts_tablet_manager.cc:1403] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260807 01:46:31.480423 27405 raft_consensus.cc:359] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.481107 27405 raft_consensus.cc:385] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:31.481380 27405 raft_consensus.cc:740] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:31.482048 27405 consensus_queue.cc:260] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.483536 27405 ts_tablet_manager.cc:1434] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 01:46:31.484256 27405 tablet_bootstrap.cc:492] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:31.494024 27405 tablet_bootstrap.cc:654] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:31.503017 27405 tablet_bootstrap.cc:492] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:31.503929 27405 ts_tablet_manager.cc:1403] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.009s
I20260807 01:46:31.510805 27405 raft_consensus.cc:359] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.511541 27405 raft_consensus.cc:385] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:31.512511 27405 raft_consensus.cc:740] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:31.513137 27405 consensus_queue.cc:260] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.515995 27405 ts_tablet_manager.cc:1434] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.012s user 0.005s sys 0.005s
I20260807 01:46:31.517071 27405 tablet_bootstrap.cc:492] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:31.522339 27405 tablet_bootstrap.cc:654] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:31.529731 27405 tablet_bootstrap.cc:492] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:31.530344 27405 ts_tablet_manager.cc:1403] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.003s
I20260807 01:46:31.532833 27405 raft_consensus.cc:359] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.533525 27405 raft_consensus.cc:385] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:31.533818 27405 raft_consensus.cc:740] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:31.534582 27405 consensus_queue.cc:260] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.536381 27405 ts_tablet_manager.cc:1434] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260807 01:46:31.537873 27405 tablet_bootstrap.cc:492] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:31.547016 27405 tablet_bootstrap.cc:654] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:31.555682 27405 tablet_bootstrap.cc:492] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:31.556222 27405 ts_tablet_manager.cc:1403] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.011s
I20260807 01:46:31.560359 27405 raft_consensus.cc:359] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.561074 27405 raft_consensus.cc:385] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:31.561371 27405 raft_consensus.cc:740] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:31.562088 27405 consensus_queue.cc:260] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.564116 27405 ts_tablet_manager.cc:1434] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.008s user 0.004s sys 0.002s
I20260807 01:46:31.565253 27405 tablet_bootstrap.cc:492] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
W20260807 01:46:31.567248 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 456705us
I20260807 01:46:31.578075 27405 tablet_bootstrap.cc:654] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:31.579058 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:31.598701 27404 tablet_bootstrap.cc:492] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:31.613821 27404 tablet_bootstrap.cc:654] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:31.632333 27063 leader_election.cc:304] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:31.633494 27930 raft_consensus.cc:2804] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:31.637984 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:31.634452 27930 raft_consensus.cc:697] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:31.642177 27930 consensus_queue.cc:237] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.659396 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:31.659471 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:31.663525 27102 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:31.664343 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:31.668196 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:31.669513 27109 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:31.671622 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:31.737633 27404 tablet_bootstrap.cc:492] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:31.738102 27404 ts_tablet_manager.cc:1403] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.140s user 0.006s sys 0.017s
I20260807 01:46:31.846133 27102 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:31.737394 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:31.891983 27110 raft_consensus.cc:1240] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260807 01:46:31.951987 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:31.952641 27099 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:31.955025 27935 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:31.960114 27935 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.962599 27935 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:31.963438 27404 raft_consensus.cc:359] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.964116 27099 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:31.964146 27404 raft_consensus.cc:385] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:31.964699 27404 raft_consensus.cc:740] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:31.965410 27404 consensus_queue.cc:260] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.967212 27404 ts_tablet_manager.cc:1434] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.229s user 0.019s sys 0.000s
I20260807 01:46:31.968240 27404 tablet_bootstrap.cc:492] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:31.968791 27934 raft_consensus.cc:493] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:31.969367 27934 raft_consensus.cc:515] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.972656 27934 leader_election.cc:290] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:31.974272 27404 tablet_bootstrap.cc:654] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:31.974799 27405 tablet_bootstrap.cc:492] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:31.975299 27405 ts_tablet_manager.cc:1403] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.410s user 0.103s sys 0.151s
I20260807 01:46:31.979163 27404 tablet_bootstrap.cc:492] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:31.979621 27404 ts_tablet_manager.cc:1403] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.001s
I20260807 01:46:31.982451 27404 raft_consensus.cc:359] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.983120 27404 raft_consensus.cc:385] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:31.983404 27404 raft_consensus.cc:740] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:31.984267 27404 consensus_queue.cc:260] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:31.986410 27404 ts_tablet_manager.cc:1434] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 01:46:31.987447 27404 tablet_bootstrap.cc:492] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:31.990823 27878 consensus_queue.cc:1048] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:31.993443 27404 tablet_bootstrap.cc:654] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
W20260807 01:46:32.000371 27036 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 01:46:32.004700 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:32.014020 27404 tablet_bootstrap.cc:492] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:32.015581 27404 ts_tablet_manager.cc:1403] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.028s user 0.012s sys 0.011s
I20260807 01:46:32.018844 27404 raft_consensus.cc:359] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.019609 27404 raft_consensus.cc:385] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:32.019894 27404 raft_consensus.cc:740] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:32.020587 27404 consensus_queue.cc:260] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.022930 27404 ts_tablet_manager.cc:1434] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 01:46:32.024260 27404 tablet_bootstrap.cc:492] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:32.045166 27951 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:32.045980 27951 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.048365 27951 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:32.049446 27404 tablet_bootstrap.cc:654] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:32.051034 27405 raft_consensus.cc:359] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.051846 27405 raft_consensus.cc:385] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:32.052157 27405 raft_consensus.cc:740] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:32.052901 27405 consensus_queue.cc:260] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.074970 27407 consensus_queue.cc:260] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.072052 27949 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:32.086592 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:32.089563 27405 ts_tablet_manager.cc:1434] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.114s user 0.010s sys 0.012s
I20260807 01:46:32.094352 27407 ts_tablet_manager.cc:1434] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.928s user 0.016s sys 0.002s
W20260807 01:46:32.094820 27407 ts_tablet_manager.cc:1467] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3: Tablet startup took 928ms
W20260807 01:46:32.095203 27407 ts_tablet_manager.cc:1469] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3: Trace:
0807 01:46:22.479123 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:23.484583 (+1005460us) service_pool.cc:224] Handling call
0807 01:46:23.572538 (+ 87955us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b5800024c98. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e0c68 00007f7d522e0b8b 00007f7d5b275159 00007f7d5a39f4e8 00007f7d5a461990 00007f7d5a410210 00007f7d53863bf4 00007f7d53863b82 00007f7d53863ab1 00007f7d53863a2c
0807 01:46:23.572617 (+ 79us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0807 01:46:23.572759 (+ 142us) ts_tablet_manager.cc:682] Creating new metadata...
0807 01:46:23.621381 (+ 48622us) tablet_metadata.cc:665] Metadata flushed
0807 01:46:23.661003 (+ 39622us) tablet_replica.cc:206] Creating consensus instance
0807 01:46:23.684868 (+ 23865us) inbound_call.cc:177] Queueing success response
0807 01:46:23.753947 (+ 69079us) spinlock_profiling.cc:243] Waited 59.6 ms on lock 0x7b4c000e1a40. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d536e7fc0 00007f7d536f8fa9 00007f7d5369ae5b 00007f7d536b1bd3 00007f7d536b1a05 00007f7d5372671d 00007f7d5a41029c 00007f7d53863bf4 00007f7d53863b82 00007f7d53863ab1 00007f7d53863a2c
0807 01:46:30.871590 (+7117643us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0807 01:46:31.165883 (+294293us) ts_tablet_manager.cc:1435] Starting tablet replica
0807 01:46:31.205606 (+ 39723us) tablet_replica.cc:261] Starting instrumentation
0807 01:46:31.206148 (+ 542us) tablet_replica.cc:280] Starting consensus
0807 01:46:32.074640 (+868492us) spinlock_profiling.cc:243] Waited 821 ms on lock 0x7b4c000e1a40. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d536e7fc0 00007f7d536f8fa9 00007f7d536be214 00007f7d536e64cd 00007f7d536e6133 00007f7d5541956b 00007f7d5541a095 00007f7d55415310 00007f7d55414020 00007f7d58e3050e 00007f7d5a464b9a
Metrics: {"raft.queue_time_us":502,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":5305,"spinlock_wait_cycles":2027842432,"tablet-open.queue_time_us":7210053}
I20260807 01:46:32.103797 27407 tablet_bootstrap.cc:492] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:32.089816 27949 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.091069 27951 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:32.096293 27804 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:32.114344 27450 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:32.114898 27450 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:32.115031 27170 leader_election.cc:343] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:32.118850 27946 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:32.119436 27946 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.123956 27450 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:32.124086 27946 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:32.125001 27450 raft_consensus.cc:493] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:32.125459 27450 raft_consensus.cc:515] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.135069 27230 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:32.121193 27804 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.156908 27407 tablet_bootstrap.cc:654] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:32.116945 27951 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.159006 27405 tablet_bootstrap.cc:492] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:32.160526 27952 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:32.168027 27450 leader_election.cc:290] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:32.161099 27952 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.198287 27941 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:32.204854 27450 consensus_queue.cc:1048] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:32.198979 27941 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.207867 27949 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:32.263572 27404 tablet_bootstrap.cc:492] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:32.264187 27404 ts_tablet_manager.cc:1403] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.240s user 0.017s sys 0.022s
I20260807 01:46:32.208878 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:32.288739 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:32.264678 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:32.386397 27450 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:32.351481 27064 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:32.443005 27450 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:32.444700 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 93341.8us
I20260807 01:46:32.448786 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:32.288166 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:32.477624 27450 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:32.350569 27887 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:32.422603 27881 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:32.295855 27952 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:32.388697 27804 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:32.319319 27951 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:32.418071 27879 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:32.290313 27336 raft_consensus.cc:2468] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:32.383507 27808 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:32.455271 27932 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:32.424265 27878 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:32.499075 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:32.455571 27956 consensus_queue.cc:1048] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.041s
I20260807 01:46:32.500890 27229 raft_consensus.cc:3060] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:32.479983 27337 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:32.476758 27752 consensus_queue.cc:1048] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
W20260807 01:46:32.512197 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 65987.2us
W20260807 01:46:32.519713 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 46825.6us
I20260807 01:46:32.544620 27404 raft_consensus.cc:359] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.545753 27404 raft_consensus.cc:385] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:32.546106 27404 raft_consensus.cc:740] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:32.546866 27404 consensus_queue.cc:260] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.549229 27404 ts_tablet_manager.cc:1434] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.285s user 0.007s sys 0.022s
I20260807 01:46:32.496140 27879 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.493466 27887 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.500514 27808 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.507907 27881 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:32.559216 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 47622.1us
I20260807 01:46:32.518749 27878 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.540460 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:32.599061 27941 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:32.599186 27965 raft_consensus.cc:2749] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:46:32.512117 27928 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:32.601866 27333 raft_consensus.cc:2468] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
W20260807 01:46:32.637712 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 123820us
I20260807 01:46:32.521693 27932 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.640997 27932 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:46:32.625237 27172 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s user 0.009s sys 0.008s
W20260807 01:46:32.642802 27172 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s user 0.009s sys 0.009s
W20260807 01:46:32.647058 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 86348.3us
W20260807 01:46:32.648056 27287 leader_election.cc:343] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:32.648489 27287 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:32.599053 27971 raft_consensus.cc:493] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
W20260807 01:46:32.676215 27964 log.cc:927] Time spent T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.276s user 0.000s sys 0.001s
I20260807 01:46:32.676918 27969 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:32.637341 27949 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:32.645589 27808 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:32.714424 27956 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:32.646255 27941 consensus_queue.cc:1059] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.070s
I20260807 01:46:32.677551 27969 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.763674 27956 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.668581 27928 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:32.828822 27141 debug-util.cc:398] Leaking SignalData structure 0x7b08000c2060 after lost signal to thread 24600
W20260807 01:46:32.832880 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 172773us
I20260807 01:46:32.713653 27949 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.893280 27887 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:32.893889 27887 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:32.894381 27887 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.896142 27966 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:32.715404 27971 raft_consensus.cc:515] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.923471 27971 leader_election.cc:290] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:46:32.928402 27170 leader_election.cc:343] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:32.929143 27170 leader_election.cc:304] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:32.934540 27984 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:32.935185 27984 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.937755 27984 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:32.940407 27172 leader_election.cc:304] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:32.941226 27991 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:32.941299 27959 raft_consensus.cc:2749] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:32.930591 27170 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:32.926618 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:32.967106 27338 raft_consensus.cc:2468] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:32.980232 27450 raft_consensus.cc:2804] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:32.980348 27172 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:32.980775 27450 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:32.981159 27450 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
W20260807 01:46:32.981192 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 50667.7us
W20260807 01:46:32.982003 27170 leader_election.cc:343] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:32.982677 27170 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
W20260807 01:46:32.990952 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 329039us
I20260807 01:46:32.896761 27966 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:32.968631 27752 raft_consensus.cc:2804] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:32.909659 27881 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:32.910696 27808 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
W20260807 01:46:32.994112 27172 leader_election.cc:343] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:32.937661 27976 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
W20260807 01:46:32.996223 27172 leader_election.cc:343] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:32.943392 27971 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:32.969359 27891 raft_consensus.cc:3055] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Stepping down as leader of term 1
W20260807 01:46:32.996862 27141 debug-util.cc:398] Leaking SignalData structure 0x7b080015ea40 after lost signal to thread 27011
I20260807 01:46:32.994724 27752 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:32.998324 27808 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:33.038365 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 68203.7us
I20260807 01:46:33.005134 27946 raft_consensus.cc:2804] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:33.002013 27975 raft_consensus.cc:2749] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:33.000680 27887 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:33.065246 27969 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:46:33.058775 27036 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 01:46:33.003716 27878 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:46:33.028889 27230 consensus_meta.cc:287] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.873s user 0.007s sys 0.003s
I20260807 01:46:33.026549 27974 raft_consensus.cc:2749] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:33.051648 27752 raft_consensus.cc:3060] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:33.028422 27949 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:33.075404 27450 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.029147 27891 raft_consensus.cc:740] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:33.056203 27946 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:33.061015 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:33.028829 27971 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:33.098111 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 97262.1us
I20260807 01:46:32.995990 27976 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } }
I20260807 01:46:33.101615 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:33.102713 27108 raft_consensus.cc:2393] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 1: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:33.103680 27976 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:33.104440 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:33.106869 27450 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:46:33.107995 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 45382us
I20260807 01:46:33.084663 27946 raft_consensus.cc:3060] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:33.083276 27879 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:46:33.093355 27141 debug-util.cc:398] Leaking SignalData structure 0x7b0800266d80 after lost signal to thread 27146
I20260807 01:46:33.112957 27108 raft_consensus.cc:2468] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:33.110519 27172 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:46:32.995852 27991 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.114554 27450 raft_consensus.cc:2804] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:33.115170 27450 raft_consensus.cc:697] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:33.117221 27172 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:33.099825 27956 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:33.118652 27450 consensus_queue.cc:237] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.115130 27891 consensus_queue.cc:260] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.118999 27976 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:33.120654 27969 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:33.110136 27974 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:33.122874 27407 tablet_bootstrap.cc:492] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:33.123317 27407 ts_tablet_manager.cc:1403] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 1.020s user 0.023s sys 0.041s
I20260807 01:46:33.112795 27887 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:33.132416 27405 tablet_bootstrap.cc:654] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
W20260807 01:46:33.133677 27172 leader_election.cc:343] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:33.136005 27404 tablet_bootstrap.cc:492] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:33.123799 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:33.120649 27966 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:33.117535 27928 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:46:33.135666 27141 debug-util.cc:398] Leaking SignalData structure 0x7b080026f9a0 after lost signal to thread 27269
I20260807 01:46:33.085990 27992 consensus_queue.cc:1048] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.124s
I20260807 01:46:33.155427 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:33.156798 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:33.157653 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:33.158196 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:33.158545 27227 raft_consensus.cc:2393] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 1: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:33.128485 27976 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.158424 27808 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:33.128326 27969 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.171293 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:33.158653 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:33.181977 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:46:33.175064 27988 log.cc:927] Time spent T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.072s user 0.000s sys 0.001s
I20260807 01:46:33.179479 27991 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:33.179029 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:33.144832 27887 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.136459 27974 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.176882 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:33.189528 27141 debug-util.cc:398] Leaking SignalData structure 0x7b080021db20 after lost signal to thread 27378
I20260807 01:46:33.168068 27752 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.191120 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:33.186726 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:33.268929 27968 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:33.266422 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:33.284454 27407 raft_consensus.cc:359] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.285254 27407 raft_consensus.cc:385] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:33.285146 28008 raft_consensus.cc:2749] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:33.285725 27407 raft_consensus.cc:740] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:33.258263 27326 raft_consensus.cc:2468] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:33.286296 27976 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:33.287118 27407 consensus_queue.cc:260] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.279780 27992 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:33.276350 27213 raft_consensus.cc:2468] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:33.304140 27407 ts_tablet_manager.cc:1434] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.181s user 0.006s sys 0.003s
I20260807 01:46:33.315624 27407 tablet_bootstrap.cc:492] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:33.318216 27064 leader_election.cc:304] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:33.321026 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:33.319224 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:33.322636 27976 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:33.323539 27976 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:33.327692 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 218155us
I20260807 01:46:33.330284 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:33.331899 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:33.334246 27974 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:33.334702 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:33.310467 27752 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:33.338898 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:33.338112 27928 raft_consensus.cc:2749] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:33.343124 27230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57662 (request call id 141) took 1246 ms. Trace:
I20260807 01:46:33.344290 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:33.345093 27404 tablet_bootstrap.cc:654] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:33.345885 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:33.346449 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:33.352689 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:33.354635 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:33.355629 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:33.356348 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:46:33.421391 27172 leader_election.cc:343] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
W20260807 01:46:33.424095 27172 leader_election.cc:343] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:33.424538 27172 leader_election.cc:304] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:33.355810 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:33.375998 27965 raft_consensus.cc:493] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:33.435768 27956 raft_consensus.cc:2749] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 01:46:33.453797 27172 leader_election.cc:343] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:33.456529 27172 leader_election.cc:304] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:33.462819 27952 raft_consensus.cc:2749] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 01:46:33.464557 27172 leader_election.cc:343] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
W20260807 01:46:33.465296 27172 leader_election.cc:343] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:33.465618 27405 tablet_bootstrap.cc:492] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
W20260807 01:46:33.466042 27172 consensus_peers.cc:597] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f -> Peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Couldn't send request to peer 3262a9d183914b699227fdba0ca5bed3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:33.466691 27405 ts_tablet_manager.cc:1403] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 1.308s user 0.034s sys 0.031s
I20260807 01:46:33.418779 27105 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:33.302204 27968 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.433279 27887 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:33.476686 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:33.314189 27992 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.482841 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:33.486526 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:33.490650 28016 raft_consensus.cc:493] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:33.470607 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:33.491734 28016 raft_consensus.cc:515] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.502915 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:33.504165 27946 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.505375 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:33.506139 27879 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:33.514285 27999 raft_consensus.cc:493] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:33.517310 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:33.510031 27992 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:33.521776 27879 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.529780 27879 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:33.534487 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:33.535789 27102 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:33.474750 27965 raft_consensus.cc:515] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.518477 27099 raft_consensus.cc:3060] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:33.554545 27999 raft_consensus.cc:515] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.551039 27878 raft_consensus.cc:493] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:33.560717 27878 raft_consensus.cc:515] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.561739 27969 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:33.575009 27226 raft_consensus.cc:2393] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 1: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:33.566382 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:33.370446 27230 rpcz_store.cc:276] 0807 01:46:32.096875 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:32.099476 (+ 2601us) service_pool.cc:224] Handling call
0807 01:46:32.121372 (+ 21896us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:46:33.091095 (+969723us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:46:33.094398 (+ 3303us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:46:33.094930 (+ 532us) log.cc:844] Serialized 26 byte log entry
0807 01:46:33.262846 (+167916us) raft_consensus.cc:1645] Marking committed up to 1
0807 01:46:33.282889 (+ 20043us) log.cc:844] Serialized 14 byte log entry
0807 01:46:33.283230 (+ 341us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:46:33.286308 (+ 3078us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:46:33.325794 (+ 39486us) raft_consensus.cc:1685] finished
0807 01:46:33.328365 (+ 2571us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:46:33.342981 (+ 14616us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35340,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":6805,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":4463,"thread_start_us":179914,"threads_started":2,"wal-append.queue_time_us":201113}
I20260807 01:46:33.607504 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:33.584419 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:33.624310 27954 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:33.624493 27968 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:33.626909 27946 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:33.631942 28000 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:33.632810 28000 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.610117 27879 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:33.661877 28021 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
W20260807 01:46:33.665982 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 199250us
W20260807 01:46:33.666875 27172 leader_election.cc:343] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
W20260807 01:46:33.668028 27172 consensus_peers.cc:597] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f -> Peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Couldn't send request to peer 3262a9d183914b699227fdba0ca5bed3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:33.655830 27965 leader_election.cc:290] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:33.669935 27405 raft_consensus.cc:359] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.670887 27407 tablet_bootstrap.cc:654] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:33.672130 27405 raft_consensus.cc:385] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:33.672475 27405 raft_consensus.cc:740] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:33.662825 27999 leader_election.cc:290] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:33.673852 27405 consensus_queue.cc:260] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.666224 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:33.684507 27405 ts_tablet_manager.cc:1434] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.217s user 0.018s sys 0.016s
I20260807 01:46:33.659726 27879 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.625216 27954 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:33.682595 28020 log.cc:927] Time spent T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.062s user 0.000s sys 0.001s
I20260807 01:46:33.662854 28021 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.672405 27946 consensus_queue.cc:1048] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260807 01:46:33.681591 28016 leader_election.cc:290] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:33.682818 28012 raft_consensus.cc:493] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:33.691851 28012 raft_consensus.cc:515] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.694793 28000 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:33.754921 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:33.754945 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:33.757612 27326 raft_consensus.cc:2468] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
I20260807 01:46:33.759621 27879 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:33.759454 27991 raft_consensus.cc:493] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:33.752908 27887 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:33.763509 27991 raft_consensus.cc:515] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.763576 27887 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.766225 27991 leader_election.cc:290] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:33.767017 27887 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:33.767585 28021 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:33.770068 27954 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:33.776762 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:33.777189 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:33.784238 28021 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:33.786239 28012 leader_election.cc:290] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:33.785187 28021 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.789285 27954 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:33.790266 27954 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.792011 28021 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:33.795655 27099 raft_consensus.cc:2468] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:33.798288 27954 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:33.800478 27971 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:33.802495 27971 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:33.803315 27971 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.806526 27971 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:33.807193 27954 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:33.808209 27954 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:33.830000 27954 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:33.837474 27287 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:33.840118 28021 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:33.847327 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:33.851562 27891 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:33.857756 27878 leader_election.cc:290] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:33.863466 27102 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:33.863472 27976 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:33.867199 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:33.851549 27405 tablet_bootstrap.cc:492] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:33.878858 28021 raft_consensus.cc:2749] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:33.892539 27405 tablet_bootstrap.cc:654] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
W20260807 01:46:34.994776 27036 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260807 01:46:35.552706 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:46:35.557054 27407 tablet_bootstrap.cc:492] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:35.559219 27407 ts_tablet_manager.cc:1403] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 2.244s user 0.023s sys 0.035s
I20260807 01:46:35.564710 27407 raft_consensus.cc:359] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.565670 27407 raft_consensus.cc:385] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:35.565994 27407 raft_consensus.cc:740] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:35.575381 27407 consensus_queue.cc:260] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.581538 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:46:35.590271 27103 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41080 (request call id 206) took 2273 ms (client timeout 2169 ms). Trace:
I20260807 01:46:35.593523 27407 ts_tablet_manager.cc:1434] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.034s user 0.010s sys 0.003s
W20260807 01:46:35.595697 27103 rpcz_store.cc:269] 0807 01:46:33.316917 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:33.320676 (+ 3759us) service_pool.cc:224] Handling call
0807 01:46:35.581021 (+2260345us) spinlock_profiling.cc:243] Waited 9.6 ms on lock 0x7b5800024c98. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e0521 00007f7d522e03d2 00007f7d5b24c219 00007f7d5a39f640 00007f7d5a467ebb 00007f7d5a467f41 00007f7d5a415eb0 00007f7d5a415794 00007f7d5395e834 00007f7d5395e7c2
0807 01:46:35.590008 (+ 8987us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22077824}
I20260807 01:46:35.606760 27407 tablet_bootstrap.cc:492] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
W20260807 01:46:35.609267 27003 debug-util.cc:398] Leaking SignalData structure 0x7b08000869a0 after lost signal to thread 24600
I20260807 01:46:35.609998 27404 tablet_bootstrap.cc:492] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:35.614862 27404 ts_tablet_manager.cc:1403] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 2.479s user 0.027s sys 0.032s
I20260807 01:46:35.626120 27404 raft_consensus.cc:359] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.640446 27404 raft_consensus.cc:385] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:35.649879 27404 raft_consensus.cc:740] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:35.652822 28044 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:35.653013 28046 raft_consensus.cc:493] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:35.653364 28040 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:35.654531 27407 tablet_bootstrap.cc:654] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:35.655869 27752 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:35.656502 28039 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:35.657047 28043 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:35.667120 27404 consensus_queue.cc:260] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.657121 28041 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
W20260807 01:46:35.654445 27891 consensus_meta.cc:287] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade: Time spent flushing consensus metadata: real 1.802s user 0.007s sys 0.016s
I20260807 01:46:35.659618 28038 raft_consensus.cc:493] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:35.662577 28049 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:35.667238 27752 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.676453 28041 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.679733 28038 raft_consensus.cc:515] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.681206 28041 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:35.681443 27752 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:35.681769 28043 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.682430 27752 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:35.684010 27891 raft_consensus.cc:493] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:35.684288 27752 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.684669 27891 raft_consensus.cc:515] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.685410 28043 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:35.681838 28038 leader_election.cc:290] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:35.664569 28040 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.692211 28040 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:35.664534 28044 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.683454 28049 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:35.700969 27170 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.659s (SENT) [suppressed 3 similar messages]
I20260807 01:46:35.701704 28044 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:35.701803 28049 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:35.703251 27752 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:35.682400 28039 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.678947 28046 raft_consensus.cc:515] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.705632 27752 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:35.704764 27891 leader_election.cc:290] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:35.709865 27752 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.711670 27404 ts_tablet_manager.cc:1434] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.096s user 0.013s sys 0.002s
I20260807 01:46:35.712320 28041 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:35.712522 28052 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:35.713186 27404 tablet_bootstrap.cc:492] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:35.712968 28041 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.713439 28049 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:35.714046 28049 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.713138 28052 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.747298 28049 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:35.747004 28040 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:35.750092 28052 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:35.749970 28040 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.752102 28046 leader_election.cc:290] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:46:35.753269 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 52437.1us
W20260807 01:46:35.754123 27170 connection.cc:256] RPC call timeout handler was delayed by -1.6051400650758296s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.188s for a response
I20260807 01:46:35.754411 28041 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:35.742769 27891 raft_consensus.cc:493] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:35.752735 28040 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:35.756554 28039 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:35.757157 28050 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:35.757134 28041 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:35.757974 28050 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.757236 27891 raft_consensus.cc:515] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.758008 28041 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:35.759611 27063 connection.cc:256] RPC call timeout handler was delayed by -1.2514423141255975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1948s for a response
W20260807 01:46:35.760010 27063 connection.cc:256] RPC call timeout handler was delayed by -1.2202306070830673s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1582s for a response
W20260807 01:46:35.760308 27063 connection.cc:256] RPC call timeout handler was delayed by -1.0825989211443812s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15990000000000001s for a response
W20260807 01:46:35.760607 27063 connection.cc:256] RPC call timeout handler was delayed by -1.0789306070655584s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17390000000000003s for a response
W20260807 01:46:35.760883 27063 connection.cc:256] RPC call timeout handler was delayed by -1.0209739999845624s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1765s for a response
W20260807 01:46:35.761165 27063 connection.cc:256] RPC call timeout handler was delayed by -1.0126833911053836s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1713s for a response
I20260807 01:46:35.763854 28041 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:35.765407 27752 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:35.758221 28048 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:35.766952 28048 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.768743 27404 tablet_bootstrap.cc:654] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:35.769456 28048 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:35.773970 28050 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:35.774899 27407 tablet_bootstrap.cc:492] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:35.775604 27407 ts_tablet_manager.cc:1403] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.170s user 0.018s sys 0.009s
I20260807 01:46:35.775553 28044 consensus_queue.cc:1048] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260807 01:46:35.785318 27407 raft_consensus.cc:359] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.786216 27407 raft_consensus.cc:385] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:35.786559 27407 raft_consensus.cc:740] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:35.854278 27752 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:35.855036 28058 raft_consensus.cc:493] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:35.855260 27752 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } }
I20260807 01:46:35.855769 28058 raft_consensus.cc:515] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.857673 27752 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:46:35.858723 27407 consensus_queue.cc:260] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.861421 27407 ts_tablet_manager.cc:1434] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.085s user 0.007s sys 0.001s
I20260807 01:46:35.863855 27407 tablet_bootstrap.cc:492] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
W20260807 01:46:35.872797 27170 connection.cc:256] RPC call timeout handler was delayed by -1.5033834809437394s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1845s for a response
W20260807 01:46:35.873188 27170 connection.cc:256] RPC call timeout handler was delayed by -1.2973400650080293s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2222s for a response
W20260807 01:46:35.873548 27170 connection.cc:256] RPC call timeout handler was delayed by -1.0164127738680691s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18230000000000002s for a response
I20260807 01:46:35.872498 27752 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:35.878510 28057 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:35.878811 27752 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.879144 28057 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.878912 27405 tablet_bootstrap.cc:492] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:35.880864 27405 ts_tablet_manager.cc:1403] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 2.030s user 0.028s sys 0.003s
I20260807 01:46:35.858074 28058 leader_election.cc:290] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:35.926184 27099 raft_consensus.cc:3060] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:35.929206 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:35.970729 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:35.971678 27104 raft_consensus.cc:2468] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:35.972393 28057 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:35.973492 27407 tablet_bootstrap.cc:654] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:35.972867 28048 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:35.974411 28048 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.966758 28058 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:35.976336 28058 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.977075 27752 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:35.977669 27752 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
W20260807 01:46:35.996793 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 128390us
I20260807 01:46:36.039556 28044 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:36.037338 28068 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:36.040307 28044 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:35.978178 27752 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.038897 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.010810 27104 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:36.042325 27212 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:36.042841 28044 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.043641 28047 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:36.058465 28074 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:36.058404 28047 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.059070 28074 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.059554 27405 raft_consensus.cc:359] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.060350 27405 raft_consensus.cc:385] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:36.060637 27405 raft_consensus.cc:740] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:36.060773 28047 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:46:36.044324 27064 connection.cc:256] RPC call timeout handler was delayed by -1.2006389580201358s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1926s for a response
I20260807 01:46:36.061645 28074 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:46:36.061939 27064 connection.cc:256] RPC call timeout handler was delayed by -1.025937344878912s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20720000000000002s for a response
I20260807 01:46:35.972958 28057 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:36.036549 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.037310 28069 raft_consensus.cc:493] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:36.063925 27404 tablet_bootstrap.cc:492] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:36.064448 27404 ts_tablet_manager.cc:1403] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.352s user 0.041s sys 0.057s
I20260807 01:46:36.064242 28069 raft_consensus.cc:515] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.064278 28057 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.067322 28047 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:36.067900 28069 leader_election.cc:290] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.068005 28057 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:35.979689 28062 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:36.069486 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:35.978252 28058 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.048127 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.069826 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.067903 28047 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.070317 27222 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:36.055195 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.071182 27404 raft_consensus.cc:359] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.063555 27752 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.011003 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.040433 28068 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.068303 28044 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:36.049544 28048 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.055441 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.082101 27101 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.069370 28062 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.082316 28044 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.071682 27110 raft_consensus.cc:1240] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260807 01:46:36.081094 27404 raft_consensus.cc:385] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:36.083799 27404 raft_consensus.cc:740] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:36.086086 28044 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.086548 28069 raft_consensus.cc:493] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:36.086557 28047 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.086863 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.087193 27752 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:36.087955 28057 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:36.087137 28069 raft_consensus.cc:515] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.088617 28057 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.089295 27891 leader_election.cc:290] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:36.083971 27405 consensus_queue.cc:260] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.096592 27404 consensus_queue.cc:260] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.099740 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.100752 27404 ts_tablet_manager.cc:1434] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.036s user 0.012s sys 0.004s
I20260807 01:46:36.098683 28058 raft_consensus.cc:493] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:36.101588 28058 raft_consensus.cc:515] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.101980 27404 tablet_bootstrap.cc:492] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:36.102463 28069 leader_election.cc:290] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.104252 28058 leader_election.cc:290] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.104632 27405 ts_tablet_manager.cc:1434] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.224s user 0.010s sys 0.015s
I20260807 01:46:36.104794 28058 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:36.088013 27752 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.091446 28048 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:36.086102 28062 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.093689 28074 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:36.121425 28074 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.124202 28074 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.119570 28048 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.145346 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:36.137494 28058 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.152060 28058 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.152943 27405 tablet_bootstrap.cc:492] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:36.155885 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.137884 27891 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:36.174860 27404 tablet_bootstrap.cc:654] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:36.176452 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.182025 27891 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.179028 28078 raft_consensus.cc:493] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:36.183431 28078 raft_consensus.cc:515] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:36.185495 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:46:36.185688 28048 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.185792 27891 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:36.176426 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.188279 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.188473 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.189265 27105 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.190052 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.190914 27100 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.191026 28078 leader_election.cc:290] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.193630 27407 tablet_bootstrap.cc:492] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:36.143568 28070 raft_consensus.cc:493] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:36.194994 27407 ts_tablet_manager.cc:1403] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.331s user 0.011s sys 0.018s
I20260807 01:46:36.195598 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.196556 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.197616 27103 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:36.201493 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.202258 27212 raft_consensus.cc:2468] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:36.202371 28070 raft_consensus.cc:515] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.204746 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.204871 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.203339 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.213693 28070 leader_election.cc:290] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:46:36.278755 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 69787.2us
I20260807 01:46:36.318451 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.333868 27405 tablet_bootstrap.cc:654] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
W20260807 01:46:36.325759 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 98172.7us
I20260807 01:46:36.331336 27407 raft_consensus.cc:359] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.338316 27407 raft_consensus.cc:385] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:36.338644 27407 raft_consensus.cc:740] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:36.339936 27407 consensus_queue.cc:260] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.216457 27105 raft_consensus.cc:2468] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.351486 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.330420 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.290642 28050 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:46:36.353737 27003 debug-util.cc:398] Leaking SignalData structure 0x7b0800081160 after lost signal to thread 27011
I20260807 01:46:36.232553 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.217900 28068 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.233522 27891 raft_consensus.cc:493] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:36.214613 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.328773 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.366129 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.361111 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.353739 28050 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.368821 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.369552 28050 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:46:36.366688 27003 debug-util.cc:398] Leaking SignalData structure 0x7b080026b8a0 after lost signal to thread 27146
W20260807 01:46:36.371474 27003 debug-util.cc:398] Leaking SignalData structure 0x7b08001d3300 after lost signal to thread 27269
W20260807 01:46:36.371781 27003 debug-util.cc:398] Leaking SignalData structure 0x7b08001d3240 after lost signal to thread 27270
I20260807 01:46:36.372237 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:46:36.372790 27003 debug-util.cc:398] Leaking SignalData structure 0x7b08000bc3a0 after lost signal to thread 27378
W20260807 01:46:36.373250 27003 debug-util.cc:398] Leaking SignalData structure 0x7b080018b440 after lost signal to thread 27450
W20260807 01:46:36.373582 27003 debug-util.cc:398] Leaking SignalData structure 0x7b08000324e0 after lost signal to thread 27878
W20260807 01:46:36.374013 27003 debug-util.cc:398] Leaking SignalData structure 0x7b08001dbd40 after lost signal to thread 27879
W20260807 01:46:36.374400 27003 debug-util.cc:398] Leaking SignalData structure 0x7b080010c720 after lost signal to thread 27965
I20260807 01:46:36.375773 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.378091 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.378772 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.385062 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.385262 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.220022 27226 raft_consensus.cc:2468] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:36.220558 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.391449 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.391739 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.393179 27216 raft_consensus.cc:2410] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260807 01:46:36.393419 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.223382 28044 consensus_queue.cc:1048] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.137s
I20260807 01:46:36.416474 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.414878 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.422000 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.427277 28057 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.396552 27891 raft_consensus.cc:515] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.430765 27407 ts_tablet_manager.cc:1434] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.235s user 0.012s sys 0.019s
I20260807 01:46:36.420352 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:46:36.427758 27172 leader_election.cc:343] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:36.432684 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.436908 27407 tablet_bootstrap.cc:492] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:36.440245 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:36.442299 28089 log.cc:927] Time spent T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.111s user 0.001s sys 0.000s
I20260807 01:46:36.450985 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.452566 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.453116 27891 leader_election.cc:290] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:36.453840 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.455132 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.460012 28093 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:36.455246 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.460790 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.460567 28093 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.461535 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.462742 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.462834 28093 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.430876 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.464018 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.464880 27404 tablet_bootstrap.cc:492] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:36.465366 27404 ts_tablet_manager.cc:1403] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.364s user 0.050s sys 0.046s
I20260807 01:46:36.459287 27172 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:36.467794 27172 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:36.469269 28048 raft_consensus.cc:2804] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:36.470952 28048 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:36.472184 28048 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:36.462015 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.475037 28078 consensus_queue.cc:1048] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260807 01:46:36.478175 27891 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:36.478866 27891 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.475615 28058 raft_consensus.cc:2804] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:36.480734 28058 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
W20260807 01:46:36.483835 27172 leader_election.cc:343] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:36.481374 28058 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:36.486653 27752 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.489488 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.491146 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.491875 27101 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:36.491991 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.492586 27226 raft_consensus.cc:2468] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:36.496508 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.497236 27218 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:36.497831 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.498570 27213 raft_consensus.cc:2468] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:36.467111 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.498610 27172 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
W20260807 01:46:36.501428 27036 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 01:46:36.500698 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.480921 27891 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:36.505182 28050 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:36.505841 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.504068 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.509207 27404 raft_consensus.cc:359] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.509608 28068 raft_consensus.cc:2804] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:36.521360 27404 raft_consensus.cc:385] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:36.512594 27405 tablet_bootstrap.cc:492] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:36.522938 27404 raft_consensus.cc:740] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:36.523269 27405 ts_tablet_manager.cc:1403] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.384s user 0.024s sys 0.019s
I20260807 01:46:36.503613 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:36.519188 27172 leader_election.cc:343] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:36.510326 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.526712 27405 raft_consensus.cc:359] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.527511 27405 raft_consensus.cc:385] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:36.527784 27405 raft_consensus.cc:740] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:36.528469 27405 consensus_queue.cc:260] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.529156 27404 consensus_queue.cc:260] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.530459 27405 ts_tablet_manager.cc:1434] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 01:46:36.531525 27405 tablet_bootstrap.cc:492] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:36.535393 28050 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.537561 28050 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:36.522905 28068 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:36.538332 28068 raft_consensus.cc:3060] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:36.545377 27407 tablet_bootstrap.cc:654] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:36.550104 27404 ts_tablet_manager.cc:1434] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.084s user 0.007s sys 0.003s
I20260807 01:46:36.551309 27404 tablet_bootstrap.cc:492] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:36.552662 28058 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.556044 26954 catalog_manager.cc:5697] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:36.555956 27064 leader_election.cc:304] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:36.558063 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.561551 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.561707 28048 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:36.562601 27287 connection.cc:256] RPC call timeout handler was delayed by -1.2854673110414296s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2792s for a response
W20260807 01:46:36.564029 27287 connection.cc:256] RPC call timeout handler was delayed by -1.2019756070803851s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19440000000000002s for a response
W20260807 01:46:36.564554 27287 connection.cc:256] RPC call timeout handler was delayed by -1.2019756070803851s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19440000000000002s for a response
I20260807 01:46:36.564061 26954 catalog_manager.cc:5697] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:36.562809 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.566852 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.563357 27227 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:36.567382 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.568501 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.569126 27333 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.570655 26954 catalog_manager.cc:5697] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:36.572530 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.573359 27405 tablet_bootstrap.cc:654] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:36.570073 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.558979 28103 raft_consensus.cc:2804] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:36.576238 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.576398 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.576792 28068 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } }
I20260807 01:46:36.577561 28103 raft_consensus.cc:493] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:36.576882 27333 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.577980 28103 raft_consensus.cc:3060] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:36.577337 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.577122 28058 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.578418 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.579355 28068 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:46:36.584036 28048 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.588333 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.589049 28103 raft_consensus.cc:515] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.590373 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.592758 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.592669 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:36.594764 27328 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:36.594295 26954 catalog_manager.cc:5697] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:36.594650 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.597433 27404 tablet_bootstrap.cc:654] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:36.602721 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:36.602810 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.603565 27108 raft_consensus.cc:2376] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1. Re-sending same reply.
I20260807 01:46:36.604524 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.605960 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.606186 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.606637 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.608044 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.608436 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.608530 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.596959 28103 leader_election.cc:290] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.612800 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:36.616981 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:36.617884 27331 raft_consensus.cc:3060] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:36.619149 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.619995 27103 raft_consensus.cc:2393] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 2: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:36.620321 27328 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.621260 27102 raft_consensus.cc:2376] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1. Re-sending same reply.
I20260807 01:46:36.622555 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.624683 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:36.625313 27326 raft_consensus.cc:3060] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:36.625777 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.633872 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.634655 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.635003 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.635361 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.635787 27222 raft_consensus.cc:2468] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:36.635982 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.636613 27337 raft_consensus.cc:2468] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.636945 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.637194 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.634624 27331 raft_consensus.cc:2468] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.638000 27327 raft_consensus.cc:2393] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 1: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:36.631683 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.630120 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.638023 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.643978 27330 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.653174 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.653013 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.654139 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.654759 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.655807 27328 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.658119 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.659075 27404 tablet_bootstrap.cc:492] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:36.658964 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.659819 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.663163 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.662341 27404 ts_tablet_manager.cc:1403] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.111s user 0.016s sys 0.002s
I20260807 01:46:36.661309 27333 raft_consensus.cc:2468] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.664239 27327 raft_consensus.cc:2468] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.661177 27407 tablet_bootstrap.cc:492] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:36.665733 27407 ts_tablet_manager.cc:1403] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.229s user 0.015s sys 0.014s
I20260807 01:46:36.664168 27103 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.669721 27405 tablet_bootstrap.cc:492] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:36.670559 27405 ts_tablet_manager.cc:1403] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.139s user 0.018s sys 0.022s
I20260807 01:46:36.673581 27407 raft_consensus.cc:359] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.683235 27407 raft_consensus.cc:385] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:36.686570 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.688732 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.688973 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:36.690091 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:36.691610 27110 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:36.690040 27328 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.692327 27891 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:36.692761 27404 raft_consensus.cc:359] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.692921 27891 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.693575 27404 raft_consensus.cc:385] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:36.693950 27404 raft_consensus.cc:740] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:36.689327 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:36.683565 27407 raft_consensus.cc:740] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:36.695554 27108 raft_consensus.cc:3060] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:36.702741 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:36.703483 27222 raft_consensus.cc:3060] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:36.704264 27172 leader_election.cc:304] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:36.704017 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.705092 27333 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.703074 27404 consensus_queue.cc:260] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.699690 27326 raft_consensus.cc:2468] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.677989 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.709117 27064 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:36.708941 27172 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:36.708396 28048 raft_consensus.cc:2804] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:36.710922 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.703178 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.711611 27328 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.699435 27105 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:36.709939 27407 consensus_queue.cc:260] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.715191 27407 ts_tablet_manager.cc:1434] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.049s user 0.004s sys 0.003s
I20260807 01:46:36.715268 28048 raft_consensus.cc:697] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:36.719259 28058 raft_consensus.cc:2804] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:36.719940 28058 raft_consensus.cc:697] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:36.719849 28048 consensus_queue.cc:237] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:36.719767 27172 leader_election.cc:343] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:36.720928 28058 consensus_queue.cc:237] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.723848 27172 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:36.726372 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.724701 27405 raft_consensus.cc:359] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.725209 27891 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:36.727442 27405 raft_consensus.cc:385] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:36.727785 27405 raft_consensus.cc:740] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:36.728577 27102 raft_consensus.cc:2468] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.729739 27405 consensus_queue.cc:260] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.730674 28068 raft_consensus.cc:2804] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:36.731086 28068 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:36.731459 28068 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:36.732164 27405 ts_tablet_manager.cc:1434] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.061s user 0.004s sys 0.003s
I20260807 01:46:36.737057 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.737435 27407 tablet_bootstrap.cc:492] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:36.737092 27110 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:46:36.741747 27405 tablet_bootstrap.cc:492] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:36.742779 28077 raft_consensus.cc:2804] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:36.743224 28077 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:36.743554 28077 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:36.752688 27222 raft_consensus.cc:2468] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:36.754590 27108 raft_consensus.cc:2468] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:46:36.758424 27891 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:36.741166 27404 ts_tablet_manager.cc:1434] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.076s user 0.008s sys 0.001s
I20260807 01:46:36.763752 28103 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:36.766928 28103 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.767416 27404 tablet_bootstrap.cc:492] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:36.767920 27891 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.769845 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.770107 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:36.770680 27326 raft_consensus.cc:2376] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1. Re-sending same reply.
I20260807 01:46:36.764154 27172 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:36.771996 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.774911 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:46:36.824429 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 60361.6us
I20260807 01:46:36.825009 27333 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.825380 27172 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:36.825176 27064 leader_election.cc:304] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:36.840871 27326 raft_consensus.cc:2468] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.841204 28048 raft_consensus.cc:2804] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:46:36.772222 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:46:36.804556 28118 log.cc:927] Time spent T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.068s user 0.000s sys 0.000s
I20260807 01:46:36.825891 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:46:36.805166 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:46:36.775782 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.777551 28071 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:36.845137 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:36.839522 27752 raft_consensus.cc:2804] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:36.775002 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.853207 27333 raft_consensus.cc:2376] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1. Re-sending same reply.
I20260807 01:46:36.777897 27105 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:46:36.803076 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.826877 27172 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:36.860317 28077 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.846555 27327 raft_consensus.cc:2468] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.813306 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.802790 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.847954 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.861788 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:36.862785 28062 raft_consensus.cc:2804] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:46:36.803399 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.847864 27331 raft_consensus.cc:2468] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.845882 28048 raft_consensus.cc:697] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:36.850701 28103 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.854568 27752 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:36.857326 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.858026 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:36.856019 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:36.812264 28068 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.858911 27407 tablet_bootstrap.cc:654] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:36.860653 27325 raft_consensus.cc:2468] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:36.844048 28093 raft_consensus.cc:2804] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:46:36.859714 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.860296 28071 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.775336 27891 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:36.904515 28057 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:36.880760 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:36.908867 28057 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.864130 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:36.910408 27326 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.909272 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:36.880817 27405 tablet_bootstrap.cc:654] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:36.913882 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.862982 27287 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:36.863396 28062 raft_consensus.cc:697] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:36.910075 28093 raft_consensus.cc:697] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:36.914752 27752 raft_consensus.cc:3060] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:36.920130 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.924435 27333 raft_consensus.cc:2468] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
W20260807 01:46:36.916913 27172 leader_election.cc:343] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:36.928148 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:36.928730 27108 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:36.933081 27108 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
W20260807 01:46:36.918183 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 55303.9us
I20260807 01:46:36.865655 27336 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:36.917976 28093 consensus_queue.cc:237] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:36.941154 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:36.918398 27891 raft_consensus.cc:2804] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:36.942184 27891 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:36.942560 27891 raft_consensus.cc:3060] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:36.917618 28062 consensus_queue.cc:237] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } }
I20260807 01:46:36.944494 27172 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:36.880002 28077 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.916987 27222 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
I20260807 01:46:36.916185 28057 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.945408 27172 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:36.917879 28068 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.918486 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.864403 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.863512 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.915162 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:36.919580 28071 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:36.922780 27330 raft_consensus.cc:2376] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1. Re-sending same reply.
I20260807 01:46:36.964644 27287 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:36.925227 27328 raft_consensus.cc:2468] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:36.989768 27064 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:36.908962 28048 consensus_queue.cc:237] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:36.989796 27172 leader_election.cc:343] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:36.994215 27287 leader_election.cc:304] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:36.947647 28058 raft_consensus.cc:2804] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:46:36.990892 28071 raft_consensus.cc:2804] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:46:36.991158 28102 raft_consensus.cc:2804] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:36.992892 28077 raft_consensus.cc:2804] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:36.991282 27327 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:46:36.990738 27338 raft_consensus.cc:2468] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:37.014263 28077 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:37.016815 28077 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:37.018607 27404 tablet_bootstrap.cc:654] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:37.030491 28058 raft_consensus.cc:697] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:37.031574 28058 consensus_queue.cc:237] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.013878 28102 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:37.053215 28102 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:37.043705 28071 raft_consensus.cc:697] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
W20260807 01:46:37.051914 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 62304.7us
I20260807 01:46:37.048133 28050 raft_consensus.cc:2804] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:37.056421 28050 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:37.056900 28050 raft_consensus.cc:3060] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
W20260807 01:46:37.056975 28122 log.cc:927] Time spent T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df: Append to log took a long time: real 0.062s user 0.000s sys 0.002s
I20260807 01:46:37.040704 27891 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.124378 27891 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:37.102571 28047 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:37.118198 28071 consensus_queue.cc:237] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.132226 28047 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.180299 28047 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:37.179903 28077 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.184096 28077 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:37.205288 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:37.215655 27227 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:37.224502 27407 tablet_bootstrap.cc:492] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:37.225063 27407 ts_tablet_manager.cc:1403] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.488s user 0.018s sys 0.004s
I20260807 01:46:37.234812 27405 tablet_bootstrap.cc:492] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:37.235278 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:37.236598 27230 raft_consensus.cc:2393] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 1: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:37.235358 27752 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.235381 27405 ts_tablet_manager.cc:1403] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.502s user 0.014s sys 0.014s
W20260807 01:46:37.239450 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 176440us
I20260807 01:46:37.245016 27287 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:37.253854 28147 raft_consensus.cc:2804] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:37.260481 27752 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:37.263851 27405 raft_consensus.cc:359] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.265015 27405 raft_consensus.cc:385] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:37.265552 27405 raft_consensus.cc:740] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:37.268484 27405 consensus_queue.cc:260] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.278323 27404 tablet_bootstrap.cc:492] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:37.278991 27404 ts_tablet_manager.cc:1403] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.512s user 0.017s sys 0.041s
I20260807 01:46:37.284011 27404 raft_consensus.cc:359] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.284780 27404 raft_consensus.cc:385] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:37.261468 28050 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.285053 27404 raft_consensus.cc:740] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:37.287010 28050 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:37.287482 27407 raft_consensus.cc:359] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.287801 27227 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:37.288273 27407 raft_consensus.cc:385] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:37.288612 27407 raft_consensus.cc:740] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:37.289247 27407 consensus_queue.cc:260] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.256356 28147 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
W20260807 01:46:37.276623 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:46:37.290380 28147 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:37.259187 28102 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.292351 27407 ts_tablet_manager.cc:1434] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.067s user 0.006s sys 0.002s
I20260807 01:46:37.309613 27407 tablet_bootstrap.cc:492] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:37.285845 27404 consensus_queue.cc:260] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:37.315434 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 108730us
I20260807 01:46:37.317955 27404 ts_tablet_manager.cc:1434] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.039s user 0.008s sys 0.000s
I20260807 01:46:37.327466 27064 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:37.328856 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:37.329558 27110 raft_consensus.cc:3060] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:37.314622 28131 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:37.338620 27405 ts_tablet_manager.cc:1434] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.099s user 0.008s sys 0.000s
I20260807 01:46:37.333498 28093 raft_consensus.cc:2804] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:37.341595 27404 tablet_bootstrap.cc:492] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:37.337178 28131 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.349637 27110 raft_consensus.cc:2468] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:46:37.351274 27172 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:37.352766 28093 raft_consensus.cc:697] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:37.338485 27101 raft_consensus.cc:1240] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:37.355317 28102 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:37.353742 28093 consensus_queue.cc:237] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.357861 28058 consensus_queue.cc:1059] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:37.358814 28058 raft_consensus.cc:3055] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:37.359135 28058 raft_consensus.cc:740] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:37.360994 28058 consensus_queue.cc:260] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.362236 28058 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:37.363945 27407 tablet_bootstrap.cc:654] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:37.351589 28131 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:37.370500 28057 consensus_queue.cc:1048] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:37.373060 28044 raft_consensus.cc:493] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:37.373628 28044 raft_consensus.cc:515] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.376547 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:37.377197 27227 raft_consensus.cc:2468] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
I20260807 01:46:37.378674 27064 leader_election.cc:304] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:37.379673 27405 tablet_bootstrap.cc:492] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:37.359437 28078 raft_consensus.cc:2804] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:46:37.379287 28044 leader_election.cc:290] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:37.368818 28147 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.387513 28155 raft_consensus.cc:2804] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:37.388648 28155 raft_consensus.cc:493] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:37.389019 28155 raft_consensus.cc:3060] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:37.385812 28078 raft_consensus.cc:697] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:37.402807 27404 tablet_bootstrap.cc:654] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:37.401999 28078 consensus_queue.cc:237] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.424969 27405 tablet_bootstrap.cc:654] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:37.411612 28155 raft_consensus.cc:515] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.429778 28155 leader_election.cc:290] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:37.430595 27227 raft_consensus.cc:1275] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:46:37.431993 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:37.433801 27216 raft_consensus.cc:3060] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:37.436691 28147 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:46:37.443203 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 194321us
I20260807 01:46:37.442551 28057 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:37.440181 28155 consensus_queue.cc:1048] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260807 01:46:37.452162 27287 leader_election.cc:304] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:37.456350 28163 raft_consensus.cc:2804] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:37.456837 28163 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:37.451608 28057 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.457255 28163 raft_consensus.cc:3060] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:37.463503 28057 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:37.466497 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:37.465309 27287 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:37.465014 26952 catalog_manager.cc:5697] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
W20260807 01:46:37.473670 27172 leader_election.cc:343] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:37.478003 27216 raft_consensus.cc:2468] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:37.463644 26954 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46138 (request call id 109) took 1583 ms. Trace:
I20260807 01:46:37.479691 26954 rpcz_store.cc:276] 0807 01:46:35.880337 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:35.882460 (+ 2123us) service_pool.cc:224] Handling call
0807 01:46:37.463559 (+1581099us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 01:46:36.734679 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 01:46:36.736221 (+ 1542us) write_op.cc:432] Acquiring schema lock in shared mode
0807 01:46:36.736273 (+ 52us) write_op.cc:435] Acquired schema lock
0807 01:46:36.736314 (+ 41us) tablet.cc:662] Decoding operations
0807 01:46:36.778489 (+ 42175us) write_op.cc:620] Acquiring the partition lock for write op
0807 01:46:36.778609 (+ 120us) write_op.cc:641] Partition lock acquired for write op
0807 01:46:36.778637 (+ 28us) tablet.cc:685] Acquiring locks for 4 operations
0807 01:46:36.779087 (+ 450us) tablet.cc:701] Row locks acquired
0807 01:46:36.779129 (+ 42us) write_op.cc:260] PREPARE: finished
0807 01:46:36.779289 (+ 160us) write_op.cc:270] Start()
0807 01:46:36.779464 (+ 175us) write_op.cc:276] Timestamp: P: 1786067196779244 usec, L: 0
0807 01:46:36.779495 (+ 31us) op_driver.cc:348] REPLICATION: starting
0807 01:46:36.801789 (+ 22294us) log.cc:844] Serialized 1669 byte log entry
0807 01:46:37.266980 (+465191us) op_driver.cc:464] REPLICATION: finished
0807 01:46:37.393400 (+126420us) write_op.cc:301] APPLY: starting
0807 01:46:37.393500 (+ 100us) tablet.cc:1367] starting BulkCheckPresence
0807 01:46:37.393718 (+ 218us) tablet.cc:1370] finished BulkCheckPresence
0807 01:46:37.393862 (+ 144us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 01:46:37.395180 (+ 1318us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 01:46:37.395493 (+ 313us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0807 01:46:37.395552 (+ 59us) write_op.cc:312] APPLY: finished
0807 01:46:37.433351 (+ 37799us) log.cc:844] Serialized 40 byte log entry
0807 01:46:37.434255 (+ 904us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:46:37.434350 (+ 95us) write_op.cc:454] Released schema lock
0807 01:46:37.434414 (+ 64us) write_op.cc:341] FINISH: Updating metrics
0807 01:46:37.434840 (+ 426us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:46:37.434867 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":21,"child_traces":[["op",{"apply.queue_time_us":112038,"apply.run_cpu_time_us":3687,"apply.run_wall_time_us":44240,"num_ops":4,"prepare.queue_time_us":87269,"prepare.run_cpu_time_us":8738,"prepare.run_wall_time_us":139613,"raft.queue_time_us":80555,"raft.run_cpu_time_us":5886,"raft.run_wall_time_us":111728,"replication_time_us":469439,"thread_start_us":212028,"threads_started":4,"wal-append.queue_time_us":126657}]]}
I20260807 01:46:37.487650 27064 leader_election.cc:304] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:37.488389 27287 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:37.481109 28163 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.491248 28147 raft_consensus.cc:2804] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:37.489270 28044 raft_consensus.cc:2804] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:37.500411 28134 raft_consensus.cc:2804] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:37.500888 28134 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:37.501250 28134 raft_consensus.cc:3060] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:37.493062 28147 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:37.518975 27287 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:37.506542 28044 raft_consensus.cc:697] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:37.524317 27404 tablet_bootstrap.cc:492] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:37.524799 27404 ts_tablet_manager.cc:1403] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.185s user 0.022s sys 0.026s
I20260807 01:46:37.530655 28134 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.532872 28134 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:37.523667 28147 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
W20260807 01:46:37.539129 27036 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 01:46:37.525072 28131 raft_consensus.cc:2804] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:37.541409 28131 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:37.541854 28131 raft_consensus.cc:3060] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:37.549108 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.550506 28044 consensus_queue.cc:237] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.564707 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.569360 28163 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:37.569597 26950 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46152 (request call id 102) took 1192 ms. Trace:
I20260807 01:46:37.573390 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.572955 26950 rpcz_store.cc:276] 0807 01:46:36.377405 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:36.377655 (+ 250us) service_pool.cc:224] Handling call
0807 01:46:37.569430 (+1191775us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1247,"reported_tablets":17}
I20260807 01:46:37.574667 26952 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46134 (request call id 106) took 1507 ms. Trace:
I20260807 01:46:37.577853 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.578179 27407 tablet_bootstrap.cc:492] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:37.579316 27407 ts_tablet_manager.cc:1403] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.270s user 0.008s sys 0.013s
I20260807 01:46:37.576484 26954 catalog_manager.cc:5697] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 1 to 2, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:37.572918 28147 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.598107 27102 raft_consensus.cc:1275] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:46:37.601257 28078 consensus_queue.cc:1048] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:37.602349 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.598387 28131 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.596817 26954 catalog_manager.cc:5697] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 1 to 2, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:37.605626 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.607761 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.608611 27331 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:37.609684 28131 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:37.611603 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.612437 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.613102 27330 raft_consensus.cc:1240] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:37.613205 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.614550 27338 raft_consensus.cc:2410] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260807 01:46:37.607961 26954 catalog_manager.cc:5697] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 1 to 2, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:37.618408 27404 raft_consensus.cc:359] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.619185 27404 raft_consensus.cc:385] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:37.619417 27404 raft_consensus.cc:740] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:37.622488 27407 raft_consensus.cc:359] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.625594 27407 raft_consensus.cc:385] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:37.625948 27407 raft_consensus.cc:740] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:37.627099 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.575055 26952 rpcz_store.cc:276] 0807 01:46:36.067154 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:36.067369 (+ 215us) service_pool.cc:224] Handling call
0807 01:46:37.574600 (+1507231us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 01:46:37.526503 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 01:46:37.527513 (+ 1010us) write_op.cc:432] Acquiring schema lock in shared mode
0807 01:46:37.527551 (+ 38us) write_op.cc:435] Acquired schema lock
0807 01:46:37.527572 (+ 21us) tablet.cc:662] Decoding operations
0807 01:46:37.528004 (+ 432us) write_op.cc:620] Acquiring the partition lock for write op
0807 01:46:37.528086 (+ 82us) write_op.cc:641] Partition lock acquired for write op
0807 01:46:37.528124 (+ 38us) tablet.cc:685] Acquiring locks for 1 operations
0807 01:46:37.528350 (+ 226us) tablet.cc:701] Row locks acquired
0807 01:46:37.528371 (+ 21us) write_op.cc:260] PREPARE: finished
0807 01:46:37.528564 (+ 193us) write_op.cc:270] Start()
0807 01:46:37.528685 (+ 121us) write_op.cc:276] Timestamp: P: 1786067197528487 usec, L: 0
0807 01:46:37.528711 (+ 26us) op_driver.cc:348] REPLICATION: starting
0807 01:46:37.529524 (+ 813us) log.cc:844] Serialized 531 byte log entry
0807 01:46:37.531374 (+ 1850us) op_driver.cc:464] REPLICATION: finished
0807 01:46:37.531734 (+ 360us) write_op.cc:301] APPLY: starting
0807 01:46:37.531806 (+ 72us) tablet.cc:1367] starting BulkCheckPresence
0807 01:46:37.531909 (+ 103us) tablet.cc:1370] finished BulkCheckPresence
0807 01:46:37.532003 (+ 94us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 01:46:37.532227 (+ 224us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 01:46:37.532373 (+ 146us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0807 01:46:37.532426 (+ 53us) write_op.cc:312] APPLY: finished
0807 01:46:37.532623 (+ 197us) log.cc:844] Serialized 22 byte log entry
0807 01:46:37.532782 (+ 159us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:46:37.532859 (+ 77us) write_op.cc:454] Released schema lock
0807 01:46:37.532895 (+ 36us) write_op.cc:341] FINISH: Updating metrics
0807 01:46:37.533245 (+ 350us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:46:37.533271 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":202927,"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":126,"apply.run_cpu_time_us":1470,"apply.run_wall_time_us":1471,"num_ops":1,"prepare.queue_time_us":37226,"prepare.run_cpu_time_us":3072,"prepare.run_wall_time_us":3364,"replication_time_us":2508,"thread_start_us":35117,"threads_started":1}]]}
I20260807 01:46:37.619961 27404 consensus_queue.cc:260] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.601202 28147 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:37.624629 26954 catalog_manager.cc:5697] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 1 to 2, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:37.631629 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.633029 27102 raft_consensus.cc:1275] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:46:37.635643 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.636513 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.638051 27404 ts_tablet_manager.cc:1434] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.113s user 0.006s sys 0.002s
I20260807 01:46:37.639555 27404 tablet_bootstrap.cc:492] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:37.642930 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.645771 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.646103 27405 tablet_bootstrap.cc:492] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:37.646603 27405 ts_tablet_manager.cc:1403] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.292s user 0.009s sys 0.008s
I20260807 01:46:37.648874 27407 consensus_queue.cc:260] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.651190 28078 consensus_queue.cc:1048] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260807 01:46:37.654300 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.654433 27103 raft_consensus.cc:1275] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:46:37.658205 27105 raft_consensus.cc:1275] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:46:37.658083 27405 raft_consensus.cc:359] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.658967 27405 raft_consensus.cc:385] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:37.659288 27405 raft_consensus.cc:740] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:37.659439 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.660017 27405 consensus_queue.cc:260] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.661393 28058 consensus_queue.cc:1048] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:37.661760 28078 consensus_queue.cc:1048] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:37.664399 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.664865 27405 ts_tablet_manager.cc:1434] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.018s user 0.003s sys 0.004s
I20260807 01:46:37.666041 27405 tablet_bootstrap.cc:492] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f: Bootstrap starting.
I20260807 01:46:37.667104 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.667403 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.667732 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.668473 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.668874 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.669312 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.669749 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:46:37.670142 27170 leader_election.cc:336] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.659s (SENT)
I20260807 01:46:37.668501 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.671211 27170 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:37.670605 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.675279 27326 raft_consensus.cc:2393] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 1: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:37.676755 26954 catalog_manager.cc:5697] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 1 to 2, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:37.680045 27407 ts_tablet_manager.cc:1434] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.100s user 0.005s sys 0.004s
I20260807 01:46:37.672173 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.679121 26954 catalog_manager.cc:5697] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:37.681749 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.677639 28078 raft_consensus.cc:2749] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:37.698990 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:46:37.700755 27170 leader_election.cc:336] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.504s (SENT)
I20260807 01:46:37.701159 27404 tablet_bootstrap.cc:654] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:37.701282 27170 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
W20260807 01:46:37.752707 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:46:37.756305 28078 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:37.770664 28153 raft_consensus.cc:2749] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:37.771449 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.775738 27407 tablet_bootstrap.cc:492] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:37.775727 28078 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.802387 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.798723 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.792377 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.807245 27337 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:37.813498 27325 raft_consensus.cc:1240] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:37.814138 28078 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:37.815155 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:37.817436 27405 tablet_bootstrap.cc:654] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:37.825169 27407 tablet_bootstrap.cc:654] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:37.826031 27404 tablet_bootstrap.cc:492] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:37.826872 27404 ts_tablet_manager.cc:1403] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.188s user 0.016s sys 0.017s
I20260807 01:46:37.833616 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:37.835456 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:46:37.836141 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:46:37.837087 27327 raft_consensus.cc:1240] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:37.836215 27325 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
W20260807 01:46:37.841730 27063 leader_election.cc:336] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.502s (SENT)
I20260807 01:46:37.842355 27063 leader_election.cc:304] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:37.844257 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.844930 27336 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
I20260807 01:46:37.846865 28155 raft_consensus.cc:2749] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:37.846894 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:37.851845 27325 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:37.858201 27404 raft_consensus.cc:359] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:37.858924 28189 log.cc:927] Time spent T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.053s user 0.000s sys 0.001s
I20260807 01:46:37.860414 27404 raft_consensus.cc:385] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:37.860939 27404 raft_consensus.cc:740] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:37.858299 28125 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:37.858250 27327 raft_consensus.cc:1240] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:37.861833 28125 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.864322 28125 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:37.866106 27404 consensus_queue.cc:260] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.877046 28084 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:37.881088 28084 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.883843 28084 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:46:37.886785 27172 leader_election.cc:343] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:37.889900 27404 ts_tablet_manager.cc:1434] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.063s user 0.010s sys 0.000s
I20260807 01:46:37.893731 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:37.894379 27336 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:37.894757 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:37.895431 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.895998 27338 raft_consensus.cc:3055] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:37.896234 27328 raft_consensus.cc:2393] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 1: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:37.896373 27338 raft_consensus.cc:740] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:37.895431 27330 raft_consensus.cc:3060] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:37.886065 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:37.900763 27110 raft_consensus.cc:1240] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:37.900913 27337 raft_consensus.cc:1240] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:37.901561 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:37.922348 27063 leader_election.cc:336] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.582s (SENT)
I20260807 01:46:37.897658 27338 consensus_queue.cc:260] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.903365 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:37.951985 27338 raft_consensus.cc:3060] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
W20260807 01:46:37.924145 27170 leader_election.cc:336] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.684s (SENT)
I20260807 01:46:37.941094 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:37.942073 27404 tablet_bootstrap.cc:492] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:37.958624 27327 raft_consensus.cc:2393] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 1: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:37.943512 27063 leader_election.cc:304] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:37.949442 27407 tablet_bootstrap.cc:492] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:37.959406 27407 ts_tablet_manager.cc:1403] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.197s user 0.014s sys 0.026s
I20260807 01:46:37.950124 27227 raft_consensus.cc:2468] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
I20260807 01:46:37.950870 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:37.950982 27405 tablet_bootstrap.cc:492] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f: No bootstrap required, opened a new log
I20260807 01:46:37.956928 27325 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:37.901351 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:46:37.995113 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 72799.1us
I20260807 01:46:37.960841 27110 raft_consensus.cc:1275] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:46:37.992113 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:37.999737 27063 leader_election.cc:336] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.599s (SENT)
I20260807 01:46:38.000180 27063 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:37.991777 27407 raft_consensus.cc:359] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:37.963229 27336 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:37.969362 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:38.007426 28057 consensus_queue.cc:1048] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:38.001735 27407 raft_consensus.cc:385] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:38.002812 28093 raft_consensus.cc:2749] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:37.991344 28084 raft_consensus.cc:2749] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.011307 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:38.012008 27222 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:37.993147 27218 raft_consensus.cc:1240] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:37.992434 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:37.992833 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:37.976298 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:46:38.023109 27170 leader_election.cc:336] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.823s (SENT)
I20260807 01:46:38.023700 27170 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.024551 28144 raft_consensus.cc:2749] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260807 01:46:38.025784 27170 leader_election.cc:336] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.824s (SENT)
I20260807 01:46:38.026336 27170 leader_election.cc:304] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.027115 28144 raft_consensus.cc:2749] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.028461 27404 tablet_bootstrap.cc:654] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:38.028836 27338 raft_consensus.cc:1275] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:46:38.032305 27230 raft_consensus.cc:2393] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 1: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:38.032177 27330 raft_consensus.cc:2468] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:37.993765 27104 raft_consensus.cc:1240] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:38.010583 27407 raft_consensus.cc:740] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:38.036571 27407 consensus_queue.cc:260] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.037742 28131 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:38.038309 28131 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:38.039150 27170 leader_election.cc:336] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.845s (SENT)
I20260807 01:46:38.039577 27170 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
W20260807 01:46:38.031616 27172 leader_election.cc:343] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:38.043720 27405 ts_tablet_manager.cc:1403] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f: Time spent bootstrapping tablet: real 0.378s user 0.025s sys 0.016s
I20260807 01:46:38.044662 28155 consensus_queue.cc:1048] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:38.042393 27064 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:38.046927 28144 raft_consensus.cc:2749] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:38.049712 28131 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:38.052489 28093 raft_consensus.cc:2804] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:38.052915 28093 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:38.053280 28093 raft_consensus.cc:3060] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
W20260807 01:46:38.053344 27063 leader_election.cc:336] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.713s (SENT)
I20260807 01:46:38.054016 27063 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.054886 28155 raft_consensus.cc:2749] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.055248 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:38.055930 27104 raft_consensus.cc:2468] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:38.056659 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:38.056857 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:38.057626 27226 raft_consensus.cc:2393] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 1: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:38.043882 26954 catalog_manager.cc:5697] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 2, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:38.067390 27108 raft_consensus.cc:1240] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:38.068158 26954 catalog_manager.cc:5697] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:38.071252 27407 ts_tablet_manager.cc:1434] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.112s user 0.004s sys 0.004s
I20260807 01:46:38.072533 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:38.072572 27407 tablet_bootstrap.cc:492] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
W20260807 01:46:38.074070 27063 leader_election.cc:336] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.732s (SENT)
I20260807 01:46:38.074890 27063 leader_election.cc:304] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
W20260807 01:46:38.078399 27063 leader_election.cc:336] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.739s (SENT)
I20260807 01:46:38.078891 27063 leader_election.cc:304] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
W20260807 01:46:38.079671 27170 leader_election.cc:336] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.880s (SENT)
I20260807 01:46:38.080106 27170 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.081272 26954 catalog_manager.cc:5697] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:38.078402 28155 raft_consensus.cc:2749] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.083711 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:38.084503 28103 raft_consensus.cc:2749] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.093468 28144 raft_consensus.cc:2749] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:38.088598 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:38.101284 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:38.107403 28144 consensus_queue.cc:1048] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:38.109347 27170 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:38.110422 28144 raft_consensus.cc:2804] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:38.110893 28144 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:38.111256 28144 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:38.110261 28155 consensus_queue.cc:1048] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:38.112967 27170 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:38.115510 27170 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.118443 27170 leader_election.cc:304] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:38.097232 28093 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.122651 27170 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.123219 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:38.125401 27216 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:38.120563 28078 raft_consensus.cc:2804] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:38.126044 28078 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:38.126472 28078 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:38.121520 27752 raft_consensus.cc:2804] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:38.127060 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:38.127221 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:38.127604 27226 raft_consensus.cc:1240] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:38.125633 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:38.128046 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:38.120782 28058 raft_consensus.cc:2749] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.124159 28153 raft_consensus.cc:2749] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.137104 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:38.138048 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:38.126309 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:38.140951 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:38.124589 27170 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.135048 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:38.135578 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:38.126648 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:38.127739 27218 raft_consensus.cc:3060] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:38.127143 27752 raft_consensus.cc:493] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:38.141304 27227 raft_consensus.cc:2468] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:38.154807 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:38.156996 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:38.157608 27227 raft_consensus.cc:3060] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:38.144388 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:38.160015 27212 raft_consensus.cc:1240] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:38.160212 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:38.153264 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:38.161062 27101 raft_consensus.cc:2393] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 1: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:38.162258 27226 raft_consensus.cc:1240] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:38.163188 27218 raft_consensus.cc:2468] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:38.164518 27226 raft_consensus.cc:1240] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:38.164775 27227 raft_consensus.cc:2468] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:38.165370 27218 raft_consensus.cc:1240] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:38.144529 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:38.167481 27063 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.169001 27063 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.169919 28155 raft_consensus.cc:2749] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:46:38.169806 28084 consensus_queue.cc:1059] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:38.155450 27752 raft_consensus.cc:3060] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:38.171850 28095 raft_consensus.cc:3055] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:38.172243 28095 raft_consensus.cc:740] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:38.172385 27405 raft_consensus.cc:359] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.173096 27405 raft_consensus.cc:385] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:38.170958 28084 raft_consensus.cc:2749] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.173381 27405 raft_consensus.cc:740] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Initialized, Role: FOLLOWER
I20260807 01:46:38.173713 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:38.174389 28095 consensus_queue.cc:260] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.175451 27063 leader_election.cc:304] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.187397 28155 raft_consensus.cc:2749] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.187791 28095 raft_consensus.cc:3060] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:38.188381 27063 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.189657 28084 raft_consensus.cc:2749] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.182135 27407 tablet_bootstrap.cc:654] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
W20260807 01:46:38.177023 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 52507.2us
I20260807 01:46:38.194252 27063 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.176980 27405 consensus_queue.cc:260] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.182361 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:38.184906 28071 raft_consensus.cc:2749] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.180472 27404 tablet_bootstrap.cc:492] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:38.197467 27170 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:38.200431 27214 raft_consensus.cc:2393] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 1: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:38.201429 27404 ts_tablet_manager.cc:1403] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.260s user 0.023s sys 0.014s
I20260807 01:46:38.201555 27063 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.203380 27063 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.205003 27405 ts_tablet_manager.cc:1434] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f: Time spent starting tablet: real 0.161s user 0.008s sys 0.000s
I20260807 01:46:38.191412 27213 raft_consensus.cc:3060] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:38.209718 27170 leader_election.cc:304] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:38.205827 28084 raft_consensus.cc:2749] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.207463 28153 raft_consensus.cc:2804] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260807 01:46:38.213915 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:46:38.202832 28155 raft_consensus.cc:2749] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.215075 27063 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.224821 28103 raft_consensus.cc:2749] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.232411 28084 raft_consensus.cc:2749] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.229911 28153 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:38.233881 28153 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:38.226691 28093 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:38.232858 27752 raft_consensus.cc:515] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.245391 27063 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.245544 27752 leader_election.cc:290] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:38.240547 28144 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.252592 27222 raft_consensus.cc:1240] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 1. Current term is 2. Ops: []
W20260807 01:46:38.253827 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 75766.9us
W20260807 01:46:38.254793 27287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:46:38.256628 27170 leader_election.cc:304] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:38.257773 27170 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:38.258461 27752 raft_consensus.cc:2804] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:38.258554 28131 raft_consensus.cc:493] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:38.258944 27752 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:38.257376 28078 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.259943 27063 leader_election.cc:304] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.259479 27752 raft_consensus.cc:3060] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:38.261075 27063 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.262342 28084 raft_consensus.cc:2749] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.263010 28084 raft_consensus.cc:2749] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:38.263536 28155 raft_consensus.cc:2749] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.258806 28077 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:38.263830 27063 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.259080 28131 raft_consensus.cc:515] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.265810 28078 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:38.266615 28144 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:38.266769 28131 leader_election.cc:290] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:38.267580 28103 raft_consensus.cc:2749] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.268316 27063 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.279892 28153 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.281253 27212 raft_consensus.cc:1240] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:38.281898 28153 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:38.270362 27752 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.264412 28077 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.273104 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:38.288712 27063 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.289353 28077 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:38.289901 28077 consensus_queue.cc:1059] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:38.290347 28103 raft_consensus.cc:2749] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.290741 28084 raft_consensus.cc:2749] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.290971 28077 raft_consensus.cc:3055] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:38.273672 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:38.275249 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:38.291340 28077 raft_consensus.cc:740] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:38.272409 28071 raft_consensus.cc:2764] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260807 01:46:38.292125 28077 consensus_queue.cc:260] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.293325 28077 raft_consensus.cc:3060] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:38.294967 27063 leader_election.cc:304] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.295755 28084 raft_consensus.cc:2749] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.296175 27063 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.300170 28093 raft_consensus.cc:2749] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.300220 27063 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.301052 28093 consensus_queue.cc:1059] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:38.276036 27170 leader_election.cc:304] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.277685 28058 raft_consensus.cc:2804] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:38.303562 27752 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:38.305742 28071 raft_consensus.cc:2749] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.309265 28084 raft_consensus.cc:2804] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:38.309515 28058 raft_consensus.cc:697] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:38.309722 28084 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:38.310127 28084 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:38.310493 28058 consensus_queue.cc:237] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:38.305632 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:46:38.303602 27170 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:38.313689 28155 consensus_queue.cc:1059] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260807 01:46:38.321609 28144 raft_consensus.cc:2804] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:38.322170 28144 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:38.322398 28155 raft_consensus.cc:3055] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:38.322808 28155 raft_consensus.cc:740] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:38.322805 27407 tablet_bootstrap.cc:492] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:38.325076 27407 ts_tablet_manager.cc:1403] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.253s user 0.021s sys 0.001s
W20260807 01:46:38.318023 27287 leader_election.cc:343] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:38.317700 27170 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:38.324352 27404 raft_consensus.cc:359] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.337603 27404 raft_consensus.cc:385] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:38.338006 27404 raft_consensus.cc:740] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:38.342123 28155 consensus_queue.cc:260] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.343575 28155 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:38.329859 28144 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:38.345777 28084 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.347013 27404 consensus_queue.cc:260] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.350344 28103 consensus_queue.cc:1059] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:38.351814 28103 raft_consensus.cc:3055] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:38.352293 28103 raft_consensus.cc:740] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:38.353163 28103 consensus_queue.cc:260] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.358413 28103 raft_consensus.cc:3060] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:38.359988 28084 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:38.361044 27213 raft_consensus.cc:2468] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:38.362087 27407 raft_consensus.cc:359] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.362799 27407 raft_consensus.cc:385] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:38.363065 27407 raft_consensus.cc:740] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:38.363688 27407 consensus_queue.cc:260] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.370316 27752 raft_consensus.cc:493] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:38.370863 27752 raft_consensus.cc:515] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.372639 27752 leader_election.cc:290] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:38.374341 26954 catalog_manager.cc:5697] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:38.375279 28153 raft_consensus.cc:2764] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260807 01:46:38.375679 27404 ts_tablet_manager.cc:1434] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.174s user 0.006s sys 0.002s
W20260807 01:46:38.376791 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 59153.5us
I20260807 01:46:38.366042 27407 ts_tablet_manager.cc:1434] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.041s user 0.006s sys 0.000s
I20260807 01:46:38.379172 27407 tablet_bootstrap.cc:492] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:38.380877 27404 tablet_bootstrap.cc:492] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade: Bootstrap starting.
I20260807 01:46:38.383989 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:38.384119 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:38.388011 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:38.389107 28144 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.392290 28144 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:38.403457 27287 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.406322 28202 raft_consensus.cc:3055] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:38.406956 28131 raft_consensus.cc:2749] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.407073 28202 raft_consensus.cc:740] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:38.414556 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:38.415474 27213 raft_consensus.cc:2468] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
I20260807 01:46:38.415865 27222 raft_consensus.cc:1240] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:38.418188 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:38.409462 28202 consensus_queue.cc:260] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.421367 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:38.422730 27099 raft_consensus.cc:2393] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 2: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:38.422915 28209 consensus_queue.cc:1059] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:38.424072 28202 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:38.425410 28209 raft_consensus.cc:3055] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Stepping down as leader of term 1
W20260807 01:46:38.430435 27172 leader_election.cc:343] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:38.436501 28209 raft_consensus.cc:740] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:38.438870 27064 leader_election.cc:304] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:38.446173 28206 raft_consensus.cc:2804] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:38.446604 28206 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:38.446974 28206 raft_consensus.cc:3060] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:38.448750 27404 tablet_bootstrap.cc:654] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:38.455714 27407 tablet_bootstrap.cc:654] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:38.457926 28084 consensus_queue.cc:1059] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:38.459547 28084 raft_consensus.cc:3055] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:38.459911 28084 raft_consensus.cc:740] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:38.461635 28131 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:38.462139 28131 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.464172 28131 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:38.468099 28084 consensus_queue.cc:260] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.469722 28084 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:38.473094 28209 consensus_queue.cc:260] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:38.480504 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 73087us
I20260807 01:46:38.481621 27287 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.486820 27099 raft_consensus.cc:1240] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:38.491339 28204 raft_consensus.cc:2804] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:38.491843 28204 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:38.492244 27752 consensus_queue.cc:1059] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:38.492901 27287 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.493227 28144 raft_consensus.cc:3055] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:38.492247 28204 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:38.500005 27287 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.494369 28144 raft_consensus.cc:740] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:38.501467 28144 consensus_queue.cc:260] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.503343 28144 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:38.506788 28131 raft_consensus.cc:2749] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.498273 28211 raft_consensus.cc:2749] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.494283 28209 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:38.523113 27407 tablet_bootstrap.cc:492] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:38.523705 27407 ts_tablet_manager.cc:1403] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.145s user 0.012s sys 0.009s
I20260807 01:46:38.526371 27404 tablet_bootstrap.cc:492] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade: No bootstrap required, opened a new log
I20260807 01:46:38.526916 27404 ts_tablet_manager.cc:1403] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade: Time spent bootstrapping tablet: real 0.150s user 0.021s sys 0.000s
I20260807 01:46:38.526207 28206 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.528282 27407 raft_consensus.cc:359] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.530311 27407 raft_consensus.cc:385] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:38.530655 27407 raft_consensus.cc:740] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:38.531543 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:38.532511 27218 raft_consensus.cc:3060] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:38.532791 27407 consensus_queue.cc:260] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.532184 28206 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:38.532505 27404 raft_consensus.cc:359] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.538895 27404 raft_consensus.cc:385] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:38.539113 27404 raft_consensus.cc:740] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Initialized, Role: FOLLOWER
I20260807 01:46:38.539827 27404 consensus_queue.cc:260] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.545581 28204 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.549197 28204 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:38.535745 27407 ts_tablet_manager.cc:1434] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.012s user 0.008s sys 0.000s
I20260807 01:46:38.561697 27407 tablet_bootstrap.cc:492] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
W20260807 01:46:38.567343 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 67363.8us
I20260807 01:46:38.569077 27287 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.572731 27404 ts_tablet_manager.cc:1434] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade: Time spent starting tablet: real 0.045s user 0.008s sys 0.000s
I20260807 01:46:38.593652 27218 raft_consensus.cc:2468] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:38.599308 28215 raft_consensus.cc:2749] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.600648 27064 leader_election.cc:304] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:38.606465 27287 leader_election.cc:304] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.611683 28202 raft_consensus.cc:2804] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:38.612485 28202 raft_consensus.cc:697] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:38.613699 28202 consensus_queue.cc:237] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.618436 28215 raft_consensus.cc:2749] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.620260 27287 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.624264 27407 tablet_bootstrap.cc:654] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:38.628585 28215 raft_consensus.cc:2749] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.630569 27287 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
W20260807 01:46:38.633797 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:46:38.634783 28204 raft_consensus.cc:2749] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.637331 27287 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.638141 28204 raft_consensus.cc:2749] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.639465 27287 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.640211 28204 raft_consensus.cc:2749] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.641582 27287 leader_election.cc:304] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.642338 28204 raft_consensus.cc:2749] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.644163 27287 leader_election.cc:304] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.644901 28204 raft_consensus.cc:2749] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.646670 27287 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.647397 28204 raft_consensus.cc:2749] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.648734 27287 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.649425 28204 raft_consensus.cc:2749] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.650087 26950 catalog_manager.cc:5697] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:38.650739 27287 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.651618 28204 raft_consensus.cc:2749] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.654501 27218 raft_consensus.cc:1240] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:38.655050 27287 leader_election.cc:304] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.657970 27287 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.660549 28204 raft_consensus.cc:2749] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.662730 27407 tablet_bootstrap.cc:492] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:38.663256 27407 ts_tablet_manager.cc:1403] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.102s user 0.015s sys 0.014s
I20260807 01:46:38.664597 28202 consensus_queue.cc:1059] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:38.664911 28204 raft_consensus.cc:2749] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.665660 27287 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.666834 28215 raft_consensus.cc:2804] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:38.666843 27287 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.667457 28215 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:38.667832 28204 raft_consensus.cc:2749] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.668263 27287 leader_election.cc:304] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.667922 28215 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:38.669149 28204 raft_consensus.cc:2749] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.670460 28209 raft_consensus.cc:3055] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:38.670970 28209 raft_consensus.cc:740] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
W20260807 01:46:38.671252 27287 leader_election.cc:343] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:38.672067 28209 consensus_queue.cc:260] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.672416 28204 consensus_queue.cc:1059] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:38.673564 28209 raft_consensus.cc:3060] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:38.673838 28211 raft_consensus.cc:3055] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:38.674480 28211 raft_consensus.cc:740] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:38.676278 28206 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:38.675890 28211 consensus_queue.cc:260] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.676915 28206 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.679745 27407 raft_consensus.cc:359] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.680498 27407 raft_consensus.cc:385] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:38.680783 27407 raft_consensus.cc:740] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:38.681052 28206 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:38.681478 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:38.681401 27407 consensus_queue.cc:260] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.682650 27218 raft_consensus.cc:2393] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 1: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:38.683914 27407 ts_tablet_manager.cc:1434] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.020s user 0.007s sys 0.000s
I20260807 01:46:38.685045 27407 tablet_bootstrap.cc:492] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:38.685472 28211 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
W20260807 01:46:38.689661 27036 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260807 01:46:38.692548 27287 leader_election.cc:343] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:38.694957 27287 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:38.694849 28204 consensus_queue.cc:1059] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:46:38.695971 28131 raft_consensus.cc:2764] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260807 01:46:38.697044 28131 consensus_queue.cc:1059] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:46:38.700721 28202 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:38.708407 28215 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.716086 28215 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:38.708441 28202 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:38.731884 27287 leader_election.cc:343] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
W20260807 01:46:38.734329 27287 leader_election.cc:343] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
W20260807 01:46:38.732543 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:46:38.735349 27287 leader_election.cc:343] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:38.732159 28215 raft_consensus.cc:3055] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Stepping down as leader of term 1
W20260807 01:46:38.736404 27287 leader_election.cc:343] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:38.733022 28131 raft_consensus.cc:3055] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:38.739189 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:38.740494 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.740900 27337 raft_consensus.cc:3055] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:38.741072 27326 raft_consensus.cc:1240] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:38.741560 27337 raft_consensus.cc:740] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:38.742624 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.743456 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:38.743628 28202 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:38.743566 27338 raft_consensus.cc:2468] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:38.736406 28215 raft_consensus.cc:740] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:38.745165 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.753010 27325 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:38.745344 27328 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:38.753515 28215 consensus_queue.cc:260] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } }
I20260807 01:46:38.754068 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.754866 27338 raft_consensus.cc:2468] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:38.755411 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.755949 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.756167 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.756610 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.756788 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.757716 27325 raft_consensus.cc:2393] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 2: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:38.742949 27337 consensus_queue.cc:260] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.747702 27101 raft_consensus.cc:3060] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:38.748718 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:38.757678 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.748754 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.759931 27333 raft_consensus.cc:2468] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:38.755105 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.760845 27337 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:38.761775 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.762032 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.763039 27333 raft_consensus.cc:3055] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:38.763350 27331 raft_consensus.cc:2393] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 1: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:38.764397 27218 raft_consensus.cc:2468] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 0.
I20260807 01:46:38.763365 27333 raft_consensus.cc:740] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:38.767155 27333 consensus_queue.cc:260] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.768213 27333 raft_consensus.cc:3060] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:38.771190 27064 leader_election.cc:304] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:38.738876 28131 raft_consensus.cc:740] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:38.754966 28215 raft_consensus.cc:3060] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:38.772848 28202 raft_consensus.cc:2804] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:38.777189 28202 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:38.777604 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.777637 28202 raft_consensus.cc:3060] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:38.779408 27338 raft_consensus.cc:2393] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 2: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
W20260807 01:46:38.781486 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:46:38.788666 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.792815 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.794135 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.794834 27331 raft_consensus.cc:2468] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:38.781970 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.799425 27407 tablet_bootstrap.cc:654] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:38.799237 27333 raft_consensus.cc:1275] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:46:38.800624 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:38.801607 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.802217 27337 raft_consensus.cc:1275] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:46:38.802898 27333 raft_consensus.cc:1240] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260807 01:46:38.804419 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:38.805123 27338 raft_consensus.cc:2393] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 2: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:38.806855 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:38.809320 27330 raft_consensus.cc:1275] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:46:38.810499 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:38.812683 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:38.812815 28202 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.815969 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.816303 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:38.817049 27218 raft_consensus.cc:3060] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:38.778227 28131 consensus_queue.cc:260] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.818015 28202 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:38.818902 28131 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:38.822381 28071 consensus_queue.cc:1048] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260807 01:46:38.822968 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:46:38.824103 27330 raft_consensus.cc:2468] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:38.826346 28215 raft_consensus.cc:493] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:38.826762 27327 raft_consensus.cc:1275] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:46:38.827231 28215 raft_consensus.cc:515] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.831705 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:38.838483 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.839372 27328 raft_consensus.cc:2468] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 0.
I20260807 01:46:38.844168 28209 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:38.844889 27218 raft_consensus.cc:2468] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:38.846813 27064 leader_election.cc:304] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:38.846886 27327 raft_consensus.cc:1240] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:38.847898 28084 consensus_queue.cc:1059] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:38.848083 28206 raft_consensus.cc:2804] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election won for term 1
W20260807 01:46:38.848733 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:46:38.849103 28206 raft_consensus.cc:697] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:38.844903 28209 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.851475 28206 consensus_queue.cc:237] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.852712 27336 raft_consensus.cc:1275] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:46:38.854882 28155 raft_consensus.cc:3055] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:38.855379 28155 raft_consensus.cc:740] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:38.855724 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:38.852978 28209 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:38.856379 28155 consensus_queue.cc:260] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.856676 27218 raft_consensus.cc:2393] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 1: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:38.858711 28155 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:38.870541 28215 leader_election.cc:290] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:38.872702 27063 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.877799 28202 consensus_queue.cc:1048] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:38.881071 28209 raft_consensus.cc:2749] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:38.882226 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:38.883397 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:38.884407 27214 raft_consensus.cc:2393] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 1: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:38.880558 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:38.885885 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:38.886178 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:38.886937 27330 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:46:38.887317 27099 raft_consensus.cc:2468] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:38.897764 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:38.899030 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:38.899426 27099 raft_consensus.cc:2368] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2.
I20260807 01:46:38.901821 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:38.903800 27407 tablet_bootstrap.cc:492] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:38.904320 27407 ts_tablet_manager.cc:1403] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.220s user 0.010s sys 0.018s
I20260807 01:46:38.905651 27170 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:38.908506 27170 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:38.909205 27752 raft_consensus.cc:2804] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:38.909567 27752 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:38.909551 27170 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:38.910071 27752 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
W20260807 01:46:38.911832 27287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:46:38.913566 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.914330 27336 raft_consensus.cc:2393] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 2: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:38.902697 27218 raft_consensus.cc:2468] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:38.915887 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:38.927170 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.927316 27287 leader_election.cc:304] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.927929 27336 raft_consensus.cc:2393] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 1: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:38.928501 27287 leader_election.cc:304] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.928673 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:38.929241 27336 raft_consensus.cc:3060] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:38.922640 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:38.925228 27407 raft_consensus.cc:359] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.931962 27407 raft_consensus.cc:385] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:38.932139 27333 raft_consensus.cc:2393] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 2: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:38.932260 27407 raft_consensus.cc:740] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:38.921566 27752 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.912353 28144 raft_consensus.cc:2804] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:38.934458 27287 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:38.924500 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:38.925544 27212 raft_consensus.cc:2410] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260807 01:46:38.937408 28144 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:38.943116 28211 raft_consensus.cc:2804] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:38.943477 28144 raft_consensus.cc:3060] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:38.935163 28131 raft_consensus.cc:2749] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:38.943725 28211 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:38.944903 28211 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
W20260807 01:46:38.933315 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:46:38.933008 28215 raft_consensus.cc:2749] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.924348 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.925899 27101 raft_consensus.cc:1240] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:38.930361 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:38.934728 27407 consensus_queue.cc:260] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.952451 27170 leader_election.cc:304] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:38.911186 28071 consensus_queue.cc:1048] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
I20260807 01:46:38.916268 27099 raft_consensus.cc:1240] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:38.924664 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.923856 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.924106 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.923432 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:38.972260 27325 raft_consensus.cc:2376] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2. Re-sending same reply.
I20260807 01:46:38.976063 27287 leader_election.cc:304] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.977391 27287 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.924597 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:38.983196 27407 ts_tablet_manager.cc:1434] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.079s user 0.005s sys 0.004s
I20260807 01:46:38.962419 27336 raft_consensus.cc:2468] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:38.966106 27752 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:38.969319 28232 raft_consensus.cc:2804] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:38.970753 27327 raft_consensus.cc:2393] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 1: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:38.971282 27337 raft_consensus.cc:2393] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 1: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:38.980475 27287 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.961589 27331 raft_consensus.cc:3060] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:38.990414 27287 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:38.992686 27407 tablet_bootstrap.cc:492] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:38.993121 28153 raft_consensus.cc:2749] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.983626 28204 raft_consensus.cc:2749] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:38.981940 28131 raft_consensus.cc:2749] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:38.975035 28211 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:38.988564 28232 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:38.991987 28221 raft_consensus.cc:2749] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:39.005188 27287 leader_election.cc:304] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:38.997391 28222 raft_consensus.cc:2804] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:39.002887 28232 raft_consensus.cc:3060] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:39.011176 28222 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:39.028168 28222 raft_consensus.cc:3060] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
W20260807 01:46:39.029177 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 76802.9us
W20260807 01:46:39.029393 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:46:39.030041 27170 leader_election.cc:304] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:39.036701 28236 raft_consensus.cc:2749] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:39.038264 27170 leader_election.cc:304] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:39.024286 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:39.039069 27099 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:39.039283 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:39.039973 28236 raft_consensus.cc:2764] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260807 01:46:39.041637 27170 leader_election.cc:304] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:39.018438 28215 raft_consensus.cc:2749] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:46:39.053884 27170 leader_election.cc:304] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:39.052706 28233 raft_consensus.cc:2749] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:39.058807 27170 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:39.060371 26950 catalog_manager.cc:5697] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:39.063473 27170 leader_election.cc:304] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:39.057682 28238 raft_consensus.cc:2749] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:39.062537 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:39.068823 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:39.072204 27407 tablet_bootstrap.cc:654] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:39.067847 28236 raft_consensus.cc:2749] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:39.075515 28144 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.077958 28211 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:39.078019 28144 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:39.083474 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:39.087507 28236 raft_consensus.cc:3060] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:39.089598 27170 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
W20260807 01:46:39.090329 27172 leader_election.cc:343] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:39.087441 27331 raft_consensus.cc:2468] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:39.093327 28232 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.093948 27099 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:39.093014 28222 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:39.096580 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 92033.2us
I20260807 01:46:39.097915 27287 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:39.098690 28211 raft_consensus.cc:2749] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:46:39.099098 27287 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:39.099653 28144 raft_consensus.cc:2749] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:39.099825 28211 raft_consensus.cc:2749] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:46:39.100086 27172 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:39.100201 27287 leader_election.cc:304] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:39.100917 28211 raft_consensus.cc:2764] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260807 01:46:39.101342 27287 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:39.104003 27287 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:39.102327 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:39.104734 28242 raft_consensus.cc:2804] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:39.105894 28242 raft_consensus.cc:697] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
W20260807 01:46:39.106279 27172 leader_election.cc:343] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:39.104574 28211 raft_consensus.cc:2749] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:46:39.106864 27287 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:39.107241 28232 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:39.104526 28222 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:39.107671 28204 raft_consensus.cc:2804] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:39.108497 28204 raft_consensus.cc:697] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:39.108742 27170 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:39.109059 27287 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:39.109319 28204 consensus_queue.cc:237] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.109835 27170 leader_election.cc:304] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:39.110376 28221 raft_consensus.cc:2749] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:46:39.111546 27287 leader_election.cc:304] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:39.112324 28221 raft_consensus.cc:2804] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:39.112825 28221 raft_consensus.cc:697] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:39.113025 28233 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:39.101670 28144 raft_consensus.cc:2804] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:39.113663 28221 consensus_queue.cc:237] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.113723 28242 consensus_queue.cc:237] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.114748 28238 raft_consensus.cc:2804] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:39.115039 28144 raft_consensus.cc:697] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:39.115188 28238 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:39.115540 28238 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:39.115916 28144 consensus_queue.cc:237] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.119289 27287 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:39.122730 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:39.123396 28206 consensus_queue.cc:1048] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:39.123458 27218 raft_consensus.cc:2468] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:39.124938 28095 consensus_queue.cc:1048] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:39.131412 27063 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:39.132529 28155 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:39.122279 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:39.124930 28211 raft_consensus.cc:2804] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:39.135344 28211 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:39.135718 28211 raft_consensus.cc:3060] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
W20260807 01:46:39.137192 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:46:39.139969 28155 raft_consensus.cc:2749] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:39.140956 28238 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.141736 27063 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:39.142105 28236 raft_consensus.cc:2749] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260807 01:46:39.150331 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:46:39.156937 28209 raft_consensus.cc:3060] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:39.157250 27063 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:39.158484 27063 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:39.160138 28206 raft_consensus.cc:2749] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:46:39.160846 28206 raft_consensus.cc:2749] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:39.163278 27063 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:39.164000 28095 raft_consensus.cc:2749] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:39.155632 28238 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:39.164901 27063 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:39.166369 28095 raft_consensus.cc:2804] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:46:39.166872 28095 raft_consensus.cc:697] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:39.167719 28095 consensus_queue.cc:237] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.170192 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:39.170807 27287 leader_election.cc:304] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:39.171950 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:39.172389 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:39.172916 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:39.173194 27218 raft_consensus.cc:2393] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 1: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:39.173698 27099 raft_consensus.cc:2393] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 1: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:39.174286 27287 leader_election.cc:304] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:39.173735 27063 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:39.175520 28206 raft_consensus.cc:2749] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 01:46:39.175940 27063 leader_election.cc:304] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:39.155534 27407 tablet_bootstrap.cc:492] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:39.176746 28206 raft_consensus.cc:3060] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:39.173517 28238 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:39.182631 28238 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.173199 28221 raft_consensus.cc:2749] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:39.185105 28209 raft_consensus.cc:2749] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:39.185920 28155 consensus_queue.cc:1048] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:39.185083 28238 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:39.187579 26950 catalog_manager.cc:5697] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:39.153007 27170 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:39.181919 27287 leader_election.cc:400] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2.
I20260807 01:46:39.194103 27287 leader_election.cc:403] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Cancelling election due to peer responding with higher term
I20260807 01:46:39.195187 28242 raft_consensus.cc:2749] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election lost for term 1. Reason: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2.
I20260807 01:46:39.121706 28233 raft_consensus.cc:2749] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:39.180796 28204 raft_consensus.cc:2749] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 01:46:39.195796 27287 leader_election.cc:343] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:39.198657 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:39.173190 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:39.203822 27287 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:39.205227 28204 raft_consensus.cc:2749] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:46:39.207425 27287 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
W20260807 01:46:39.207959 27172 leader_election.cc:343] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:39.176852 27407 ts_tablet_manager.cc:1403] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.184s user 0.013s sys 0.003s
I20260807 01:46:39.209136 28233 raft_consensus.cc:2749] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260807 01:46:39.209167 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 56188.4us
I20260807 01:46:39.214134 27170 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:39.214677 28204 raft_consensus.cc:2749] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:46:39.217796 28206 raft_consensus.cc:2749] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:39.214439 27407 raft_consensus.cc:359] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.219719 27407 raft_consensus.cc:385] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:39.220278 27407 raft_consensus.cc:740] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:39.221001 27170 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
W20260807 01:46:39.223011 27287 leader_election.cc:343] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:39.223780 27170 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:39.225095 27170 leader_election.cc:304] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:39.222618 28144 raft_consensus.cc:2749] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:39.222723 28233 raft_consensus.cc:2749] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:46:39.227658 28236 consensus_queue.cc:1048] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260807 01:46:39.229454 27287 leader_election.cc:343] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:39.229614 28232 raft_consensus.cc:2749] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:46:39.222615 28238 consensus_queue.cc:1048] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260807 01:46:39.232208 27407 consensus_queue.cc:260] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.234848 27407 ts_tablet_manager.cc:1434] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.025s user 0.007s sys 0.001s
I20260807 01:46:39.235945 27407 tablet_bootstrap.cc:492] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:39.192986 28211 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.237576 27170 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:39.243511 27287 leader_election.cc:304] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:39.244659 27287 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:39.240626 26952 catalog_manager.cc:5697] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:39.247735 26952 catalog_manager.cc:5697] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:39.248842 27287 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:39.249429 26952 catalog_manager.cc:5697] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 0 to 1, leader changed from <none> to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:39.255121 27170 leader_election.cc:304] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:39.254272 28221 raft_consensus.cc:2749] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 01:46:39.255896 28232 raft_consensus.cc:2749] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:46:39.239012 28238 raft_consensus.cc:2749] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:46:39.257138 28211 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:39.257408 27170 leader_election.cc:304] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:39.253410 28204 raft_consensus.cc:2804] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:39.261701 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:39.262486 28204 raft_consensus.cc:493] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:39.262964 28204 raft_consensus.cc:3060] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:39.255009 28144 raft_consensus.cc:2749] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:46:39.254628 28222 raft_consensus.cc:2749] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 01:46:39.260630 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:39.261294 28232 consensus_queue.cc:1048] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:39.266901 27212 raft_consensus.cc:2393] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 1: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:39.259017 28233 consensus_queue.cc:1048] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260807 01:46:39.254701 28209 consensus_queue.cc:1048] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260807 01:46:39.258510 28238 raft_consensus.cc:2804] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 01:46:39.269717 26954 catalog_manager.cc:5697] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 0 to 2, leader changed from <none> to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:39.273736 28238 raft_consensus.cc:493] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:39.279240 28238 raft_consensus.cc:3060] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:39.280397 28084 consensus_queue.cc:1048] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 01:46:39.270584 27287 leader_election.cc:343] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:39.289319 27287 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:39.292042 28222 raft_consensus.cc:2749] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 01:46:39.298514 28206 consensus_queue.cc:1048] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:39.277273 28204 raft_consensus.cc:515] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.313455 28238 raft_consensus.cc:515] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.318022 27407 tablet_bootstrap.cc:654] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:39.318084 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:46:39.323576 27172 leader_election.cc:343] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:39.324852 28238 leader_election.cc:290] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:39.330933 28204 leader_election.cc:290] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:39.332942 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:39.335908 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:39.337275 27212 raft_consensus.cc:2393] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 1: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
W20260807 01:46:39.337883 27287 leader_election.cc:343] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Illegal state: must be running to vote when last-logged opid is not known
I20260807 01:46:39.338835 27287 leader_election.cc:304] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:39.341120 28222 raft_consensus.cc:2749] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260807 01:46:39.356321 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 43943us
I20260807 01:46:39.358175 27170 leader_election.cc:304] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:39.358955 28238 raft_consensus.cc:2764] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260807 01:46:39.361292 28238 consensus_queue.cc:1059] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:39.362596 28233 raft_consensus.cc:3055] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:39.362952 28233 raft_consensus.cc:740] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:39.363768 28233 consensus_queue.cc:260] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.367856 28233 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:39.378816 28238 consensus_queue.cc:1048] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:39.381618 28238 consensus_queue.cc:1048] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:39.384018 27407 tablet_bootstrap.cc:492] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:39.384603 27407 ts_tablet_manager.cc:1403] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.149s user 0.013s sys 0.007s
I20260807 01:46:39.385699 27170 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:39.387617 27407 raft_consensus.cc:359] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.388343 27407 raft_consensus.cc:385] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:39.388906 28233 raft_consensus.cc:2804] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:46:39.388942 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:39.389629 27407 raft_consensus.cc:740] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:39.405882 27328 raft_consensus.cc:2393] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 1: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:39.405948 27407 consensus_queue.cc:260] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.407670 27407 ts_tablet_manager.cc:1434] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.023s user 0.007s sys 0.001s
I20260807 01:46:39.408679 27407 tablet_bootstrap.cc:492] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:39.409163 28233 raft_consensus.cc:697] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:39.414587 28233 consensus_queue.cc:237] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.420985 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:39.431429 27331 raft_consensus.cc:2393] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 1: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:39.421782 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:39.432600 27337 raft_consensus.cc:2393] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 1: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:39.422487 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:39.433432 27327 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:39.423359 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:39.434722 27333 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:46:39.439419 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:39.440204 27337 raft_consensus.cc:2393] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 1: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:39.492249 27407 tablet_bootstrap.cc:654] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:39.502279 28095 raft_consensus.cc:493] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:39.504793 28095 raft_consensus.cc:515] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.568675 28206 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:39.571830 28206 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.574110 28206 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:39.580030 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:39.581036 27336 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:39.583412 27063 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:39.592137 27327 raft_consensus.cc:2468] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:39.592983 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:39.610411 27170 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:39.598913 28206 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:39.616528 27407 tablet_bootstrap.cc:492] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:39.616256 28206 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.617053 27407 ts_tablet_manager.cc:1403] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.209s user 0.012s sys 0.011s
I20260807 01:46:39.625767 28291 raft_consensus.cc:2804] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:39.625952 28209 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:39.626361 28291 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:39.626909 28291 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:39.626761 28209 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.629132 28209 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:46:39.629940 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 46621.1us
I20260807 01:46:39.633679 28291 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.636068 27407 raft_consensus.cc:359] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.645254 27407 raft_consensus.cc:385] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:39.645623 27407 raft_consensus.cc:740] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:39.643726 27170 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:39.609579 27214 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:39.647022 27170 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:39.647675 27407 consensus_queue.cc:260] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.651556 27170 leader_election.cc:304] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:39.652102 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:39.618070 28211 consensus_queue.cc:1048] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.033s
I20260807 01:46:39.646416 28233 raft_consensus.cc:2749] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 01:46:39.652021 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:39.663085 26949 catalog_manager.cc:5697] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 1 to 2, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:39.667551 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:39.668530 27214 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:39.655202 28153 raft_consensus.cc:2804] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:39.669345 28153 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:39.669979 28153 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:39.674718 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:39.675931 27222 raft_consensus.cc:2393] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 1: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:39.649194 28144 raft_consensus.cc:2749] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 01:46:39.656431 28071 raft_consensus.cc:2749] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 01:46:39.679579 28206 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:39.683478 28221 consensus_queue.cc:1048] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:39.684262 27407 ts_tablet_manager.cc:1434] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.067s user 0.004s sys 0.003s
I20260807 01:46:39.654801 28095 leader_election.cc:290] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:39.690222 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:39.690933 27336 raft_consensus.cc:2468] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:39.684410 27212 raft_consensus.cc:2393] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 1: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:39.664075 28291 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:39.694622 27407 tablet_bootstrap.cc:492] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3: Bootstrap starting.
I20260807 01:46:39.709887 28144 consensus_queue.cc:1048] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260807 01:46:39.713732 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 62241.6us
I20260807 01:46:39.715481 27170 leader_election.cc:304] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:39.716615 27752 raft_consensus.cc:2804] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 01:46:39.717141 27752 raft_consensus.cc:697] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:39.719825 28206 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:39.720633 28206 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.720055 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:39.725399 27337 raft_consensus.cc:2393] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 1: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:39.721959 27752 consensus_queue.cc:237] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.735209 27063 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:39.733323 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:39.743432 27337 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:39.744850 28155 raft_consensus.cc:2804] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:39.745729 28155 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:39.746529 28155 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:39.749816 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:39.750775 27338 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:39.745450 27328 raft_consensus.cc:1275] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260807 01:46:39.755512 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:39.756397 27212 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:39.771231 27064 leader_election.cc:304] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:39.779008 27064 leader_election.cc:304] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:39.786643 28211 consensus_queue.cc:1048] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:39.788870 27337 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:46:39.762099 28206 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:39.779937 28289 raft_consensus.cc:3060] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:39.781244 28071 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:39.791102 28291 raft_consensus.cc:2804] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:39.789757 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:39.781421 28221 consensus_queue.cc:1048] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260807 01:46:39.816068 27063 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:39.782593 28242 consensus_queue.cc:1048] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.033s
I20260807 01:46:39.822260 28242 consensus_queue.cc:1048] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:39.815021 28291 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:39.840754 27214 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
W20260807 01:46:39.841382 27036 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 01:46:39.863646 28206 raft_consensus.cc:2804] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:39.851696 27330 raft_consensus.cc:2393] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 1: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:39.842916 28291 raft_consensus.cc:3060] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:39.816265 28071 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.828100 28290 consensus_queue.cc:1048] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.046s
I20260807 01:46:39.901062 28155 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.888537 28206 raft_consensus.cc:697] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:39.900532 28095 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:39.932241 28153 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.933624 27407 tablet_bootstrap.cc:654] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3: Neither blocks nor log segments found. Creating new log.
I20260807 01:46:39.935653 28071 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:39.943567 28155 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:39.942791 28095 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.939128 28206 consensus_queue.cc:237] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.947217 28095 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:39.951505 27218 raft_consensus.cc:1240] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 2. Current term is 3. Ops: []
I20260807 01:46:39.957221 28289 raft_consensus.cc:2749] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:39.959452 28153 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:39.969149 28291 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:39.979743 28291 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:46:39.995637 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 200752us
W20260807 01:46:40.005180 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 186204us
I20260807 01:46:40.005781 28289 raft_consensus.cc:493] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:40.008178 28289 raft_consensus.cc:515] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:40.005427 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 116162us
I20260807 01:46:40.013661 27063 leader_election.cc:304] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:40.043766 28310 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:40.044265 28222 raft_consensus.cc:493] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:40.061372 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:40.062093 27229 raft_consensus.cc:2468] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:40.076071 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:40.076954 27104 raft_consensus.cc:2468] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:46:40.078136 27108 raft_consensus.cc:1275] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:46:40.057549 28222 raft_consensus.cc:515] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.088135 28222 leader_election.cc:290] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:40.088016 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:40.088423 28155 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:40.088856 27213 raft_consensus.cc:3060] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:40.089097 28155 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:40.092373 28312 log.cc:927] Time spent T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.081s user 0.002s sys 0.000s
I20260807 01:46:40.092767 28155 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:40.095717 27172 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:40.101042 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:40.101840 27333 raft_consensus.cc:2468] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:40.102872 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:40.103464 27333 raft_consensus.cc:3060] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:40.105899 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:40.107069 27327 raft_consensus.cc:2468] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:40.105947 27407 tablet_bootstrap.cc:492] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3: No bootstrap required, opened a new log
I20260807 01:46:40.110459 27407 ts_tablet_manager.cc:1403] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3: Time spent bootstrapping tablet: real 0.416s user 0.010s sys 0.011s
I20260807 01:46:40.112977 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:40.114058 28144 consensus_queue.cc:1048] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:40.116042 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:40.117470 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:40.113003 28303 raft_consensus.cc:2804] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:40.126566 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:40.127763 27337 raft_consensus.cc:2393] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 2: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:40.129791 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:40.131191 27337 raft_consensus.cc:2468] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:40.071733 28289 leader_election.cc:290] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:40.079638 28290 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:40.120093 28206 consensus_queue.cc:1059] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260807 01:46:40.135043 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:40.133570 27064 leader_election.cc:304] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:40.139081 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:40.139698 27104 raft_consensus.cc:2468] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:46:40.122188 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:40.142133 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:40.113799 27230 raft_consensus.cc:2468] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:40.118991 28303 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:40.147465 26954 catalog_manager.cc:5697] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:40.125883 27212 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:40.151659 27063 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:40.152789 27287 leader_election.cc:304] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:40.152648 28303 raft_consensus.cc:3060] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:40.154803 27326 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:40.160941 28206 consensus_queue.cc:1243] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer 3478729505b046c2a0051c2d0130873f log is divergent from this leader: its last log entry 1.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20260807 01:46:40.138278 28290 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.155859 28233 consensus_queue.cc:1048] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260807 01:46:40.165261 28289 raft_consensus.cc:2804] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:40.167150 28202 raft_consensus.cc:2804] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:40.200459 28290 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:40.163300 28304 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:40.221524 28304 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.188530 27407 raft_consensus.cc:359] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.224692 27407 raft_consensus.cc:385] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 01:46:40.225031 27407 raft_consensus.cc:740] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Initialized, Role: FOLLOWER
I20260807 01:46:40.211249 27338 raft_consensus.cc:1275] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:46:40.212491 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:40.237138 27214 raft_consensus.cc:2393] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 2: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:40.215420 28155 raft_consensus.cc:3055] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:46:40.238564 28155 raft_consensus.cc:740] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:40.205327 28202 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:40.243991 27333 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:46:40.219236 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:40.228266 28304 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:40.229717 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:40.230463 27407 consensus_queue.cc:260] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.236110 28310 raft_consensus.cc:2749] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:40.165076 28211 raft_consensus.cc:2804] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:40.207037 28289 raft_consensus.cc:493] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:40.263731 27407 ts_tablet_manager.cc:1434] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3: Time spent starting tablet: real 0.153s user 0.010s sys 0.003s
I20260807 01:46:40.245810 28291 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:40.247908 28202 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:40.262755 28289 raft_consensus.cc:3060] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:40.260022 28211 raft_consensus.cc:493] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:40.318270 27222 raft_consensus.cc:2393] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 1: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:40.340338 28306 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:40.321045 28211 raft_consensus.cc:3060] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:40.273105 28291 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.397064 28291 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:40.369346 28306 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.414522 27222 raft_consensus.cc:1275] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:40.418119 28306 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:40.420009 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:40.421082 27214 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:40.261641 28155 consensus_queue.cc:260] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.422739 28155 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:40.437146 28321 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:40.437855 28321 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.453025 27064 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:40.457954 28095 consensus_queue.cc:1048] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:40.462867 28304 raft_consensus.cc:2804] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:40.479293 28304 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:40.472927 27213 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
W20260807 01:46:40.518133 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 352602us
I20260807 01:46:40.500193 28304 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:40.521103 27064 leader_election.cc:304] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:40.516536 28290 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:46:40.537899 27063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:46:40.537818 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 361127us
I20260807 01:46:40.540220 27063 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:40.527006 28310 raft_consensus.cc:2804] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:40.543287 27170 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:40.559340 27752 raft_consensus.cc:2749] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:46:40.556380 28310 raft_consensus.cc:697] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:40.562028 28310 consensus_queue.cc:237] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.565368 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:40.566226 27333 raft_consensus.cc:2393] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 1: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:40.570336 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:40.571120 27213 raft_consensus.cc:2393] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 2: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:40.546540 28206 raft_consensus.cc:2804] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:40.581166 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:40.537900 28290 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.569993 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:40.601440 27330 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:40.601325 28211 raft_consensus.cc:515] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.603456 28211 leader_election.cc:290] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:40.556066 28235 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:40.568148 28306 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:40.604764 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:46:40.634979 28327 log.cc:927] Time spent T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.106s user 0.001s sys 0.000s
I20260807 01:46:40.636235 28238 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:40.658645 27326 raft_consensus.cc:2468] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:46:40.659073 27212 raft_consensus.cc:2468] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:46:40.660858 28303 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.661747 28306 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.580801 28206 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:40.555886 28202 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:40.669993 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 126310us
W20260807 01:46:40.674243 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 107245us
I20260807 01:46:40.688210 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:40.689005 27230 raft_consensus.cc:2468] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:40.692378 27064 leader_election.cc:304] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:40.693733 27226 raft_consensus.cc:2468] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:40.707463 27336 raft_consensus.cc:2410] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260807 01:46:40.676662 28206 raft_consensus.cc:3060] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:40.698225 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:40.668876 28238 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.662987 28235 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.716123 28310 raft_consensus.cc:2804] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:40.743474 28310 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:40.747370 28310 raft_consensus.cc:3060] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
W20260807 01:46:40.740878 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 48572.6us
I20260807 01:46:40.724429 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
W20260807 01:46:40.754235 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 64898.7us
I20260807 01:46:40.760753 27063 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:40.773134 27063 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:40.776960 28235 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:40.779407 27230 raft_consensus.cc:2393] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 2: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:40.799163 28335 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:40.810170 28303 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:46:40.820410 28340 log.cc:927] Time spent T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.151s user 0.000s sys 0.001s
I20260807 01:46:40.826293 28331 consensus_queue.cc:1048] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.156s
I20260807 01:46:40.830672 28341 raft_consensus.cc:3060] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:40.783486 28155 raft_consensus.cc:2804] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:40.847080 28345 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:40.807817 28242 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:40.859072 28155 raft_consensus.cc:697] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:40.880297 28155 consensus_queue.cc:237] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.860805 28351 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:40.788393 28289 raft_consensus.cc:515] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.887655 28350 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:40.892103 28238 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:40.878620 28304 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.897027 28304 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:40.898415 27099 raft_consensus.cc:2368] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3.
I20260807 01:46:40.867143 28306 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:46:40.902617 28122 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.912s user 0.047s sys 0.052s
W20260807 01:46:40.903180 28122 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.913s user 0.047s sys 0.053s
I20260807 01:46:40.878981 28242 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.909354 28242 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:40.883299 28351 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.800494 28335 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.902359 28202 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:40.923969 28351 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:40.847846 28345 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.928347 27100 raft_consensus.cc:2368] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3.
I20260807 01:46:40.934617 28345 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:40.936053 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:40.937156 27213 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:40.938475 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:40.939211 27212 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:40.888530 28350 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:40.955222 27064 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:40.957684 28350 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:40.960364 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:40.961320 27213 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:40.968604 28209 raft_consensus.cc:2804] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:40.969338 28209 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:40.969960 28209 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:40.971793 28335 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:40.980476 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:40.981307 27227 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:40.992197 28290 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:40.993815 28321 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:40.997368 28289 leader_election.cc:290] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:41.000717 28310 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:41.012982 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:41.013760 27226 raft_consensus.cc:3060] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:41.019538 27064 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
W20260807 01:46:41.020150 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 327749us
I20260807 01:46:41.014523 28335 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:41.016177 28351 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:41.022451 28291 raft_consensus.cc:2804] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:41.016429 28206 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:41.044219 28291 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:41.026693 26949 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46138 (request call id 121) took 1419 ms. Trace:
W20260807 01:46:41.054652 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 345225us
W20260807 01:46:41.063047 27063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:46:41.039934 28351 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:41.053416 28204 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:41.077111 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:41.077991 27333 raft_consensus.cc:2393] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 2: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:41.080335 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:41.083340 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:41.084271 27227 raft_consensus.cc:2410] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260807 01:46:41.086722 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:41.087394 27227 raft_consensus.cc:2468] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:41.073328 28347 consensus_queue.cc:1048] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260807 01:46:41.073822 28204 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:41.092028 28204 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:41.093828 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:41.094642 27227 raft_consensus.cc:2393] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 2: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:41.096057 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:41.094638 28351 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:41.096905 27331 raft_consensus.cc:2393] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 2: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:41.097153 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:41.097795 27227 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:41.056113 28291 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:41.103528 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:41.104830 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:41.105424 27110 raft_consensus.cc:2468] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 0.
I20260807 01:46:41.106894 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:41.107537 27326 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 3
I20260807 01:46:41.107796 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:41.108276 27063 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:41.108422 27325 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:41.109480 28341 raft_consensus.cc:2749] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:41.110231 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:41.111419 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:41.113319 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:41.114050 27333 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:46:41.115399 27338 raft_consensus.cc:1275] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:41.116428 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:41.119524 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:41.120394 27333 raft_consensus.cc:2393] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 1: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:41.044871 28335 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:41.116992 27338 raft_consensus.cc:2468] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:41.110873 27325 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:41.125274 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:41.120221 28321 raft_consensus.cc:2749] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:46:41.160105 27333 raft_consensus.cc:3060] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:41.161911 27336 raft_consensus.cc:1275] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260807 01:46:41.174755 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 61154.4us
I20260807 01:46:41.174376 28238 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:41.203438 28238 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:41.199009 28310 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:41.212246 27287 leader_election.cc:400] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3.
I20260807 01:46:41.155318 28221 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:41.212862 27287 leader_election.cc:403] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260807 01:46:41.242079 28144 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:41.246387 28144 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:41.246848 28221 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:41.255195 27333 raft_consensus.cc:2468] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:46:41.265746 27226 raft_consensus.cc:2468] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:46:41.204587 28206 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:46:41.281780 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 69749.5us
I20260807 01:46:41.276099 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:41.277972 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:41.282801 27287 leader_election.cc:400] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3.
I20260807 01:46:41.283015 27110 raft_consensus.cc:2368] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3.
I20260807 01:46:41.283106 27287 leader_election.cc:403] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260807 01:46:41.258509 28303 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:41.287915 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:41.292614 28206 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:41.293073 27212 raft_consensus.cc:2468] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:46:41.293231 28206 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:41.295141 28338 log.cc:927] Time spent T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.569s user 0.003s sys 0.009s
I20260807 01:46:41.295435 28206 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:41.295933 27213 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:46:41.297075 28335 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:41.298346 28335 raft_consensus.cc:2749] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3.
I20260807 01:46:41.299504 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:41.300583 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:41.301242 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:41.301987 27100 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:41.300724 27287 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:41.297407 28221 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:41.313621 27170 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:41.314380 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:41.315357 27230 raft_consensus.cc:2393] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 1: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:41.318760 27063 leader_election.cc:304] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:41.320250 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:41.320664 27287 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:41.321964 27226 raft_consensus.cc:1275] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:41.318725 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:41.322875 27229 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
W20260807 01:46:41.326906 28348 log.cc:927] Time spent T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.598s user 0.004s sys 0.005s
I20260807 01:46:41.303325 27218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57654 (request call id 283) took 1192 ms. Trace:
I20260807 01:46:41.330238 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:41.330981 27336 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:41.289155 28306 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 3
I20260807 01:46:41.332182 27287 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:41.333305 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:41.338035 27064 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:41.335355 28310 raft_consensus.cc:2804] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:41.328128 27218 rpcz_store.cc:276] 0807 01:46:40.110979 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:40.111522 (+ 543us) service_pool.cc:224] Handling call
0807 01:46:40.111967 (+ 445us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:46:40.136760 (+ 24793us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:46:40.138649 (+ 1889us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:46:40.175064 (+ 36415us) log.cc:844] Serialized 26 byte log entry
0807 01:46:40.574404 (+399340us) raft_consensus.cc:1645] Marking committed up to 0
0807 01:46:40.575308 (+ 904us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:46:40.579753 (+ 4445us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:46:41.294353 (+714600us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b440002250c. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d5223e855 00007f7d5223f272 00007f7d54fc93bb 00007f7d54fc4c9f 00007f7d54fb8a5e 00007f7d54fb7da6 00007f7d54fb79c8 00007f7d54fc1411 00007f7d54fc13c9 00007f7d54fc1359
0807 01:46:41.302661 (+ 8308us) raft_consensus.cc:1685] finished
0807 01:46:41.302686 (+ 25us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:46:41.303143 (+ 457us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18450,"raft.queue_time_us":23128,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":14490,"spinlock_wait_cycles":26448256,"thread_start_us":338533,"threads_started":1,"wal-append.queue_time_us":496298}
I20260807 01:46:41.332753 28155 consensus_queue.cc:1048] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260807 01:46:41.345393 28310 raft_consensus.cc:697] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:41.316170 28321 consensus_queue.cc:1048] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:41.347139 27064 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:41.290620 27326 raft_consensus.cc:2468] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:46:41.309947 28144 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:46:41.335057 28366 log.cc:927] Time spent T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.166s user 0.000s sys 0.001s
I20260807 01:46:41.348071 27287 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:41.312403 28335 raft_consensus.cc:2804] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:41.353169 28206 raft_consensus.cc:2804] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:41.429291 28310 consensus_queue.cc:237] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:41.347035 28233 raft_consensus.cc:2804] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:41.433307 28209 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:41.429426 28354 raft_consensus.cc:2804] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:41.395069 28370 raft_consensus.cc:2804] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:41.306337 28303 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:41.439203 27752 raft_consensus.cc:493] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:46:41.439401 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 91392us
I20260807 01:46:41.434676 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:41.426442 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:41.443243 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 86200.6us
I20260807 01:46:41.444489 28367 raft_consensus.cc:2804] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:41.444926 28328 raft_consensus.cc:2749] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260807 01:46:41.445693 28381 log.cc:927] Time spent T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.064s user 0.000s sys 0.000s
I20260807 01:46:41.453517 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:41.458278 28355 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:41.459944 28209 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:41.460700 27110 raft_consensus.cc:2368] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3.
I20260807 01:46:41.439890 28335 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:41.817534 28335 raft_consensus.cc:3060] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:41.476194 27752 raft_consensus.cc:515] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:41.518683 27064 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:41.519152 28242 raft_consensus.cc:493] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:41.456223 28233 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:41.799068 28153 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:41.801071 27229 raft_consensus.cc:3060] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:41.526536 27230 raft_consensus.cc:3060] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:41.475032 28206 raft_consensus.cc:697] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:41.805177 27328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59490 (request call id 377) took 1240 ms. Trace:
I20260807 01:46:41.551849 28367 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:41.517602 28370 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:41.589736 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:41.443671 28291 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:41.726469 28384 log.cc:927] Time spent T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.069s user 0.001s sys 0.000s
I20260807 01:46:41.055202 26949 rpcz_store.cc:276] 0807 01:46:39.605315 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:39.614511 (+ 9196us) service_pool.cc:224] Handling call
0807 01:46:41.024679 (+1410168us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 01:46:39.986111 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 01:46:39.986694 (+ 583us) write_op.cc:432] Acquiring schema lock in shared mode
0807 01:46:39.986732 (+ 38us) write_op.cc:435] Acquired schema lock
0807 01:46:39.986750 (+ 18us) tablet.cc:662] Decoding operations
0807 01:46:39.987127 (+ 377us) write_op.cc:620] Acquiring the partition lock for write op
0807 01:46:39.987213 (+ 86us) write_op.cc:641] Partition lock acquired for write op
0807 01:46:39.987249 (+ 36us) tablet.cc:685] Acquiring locks for 1 operations
0807 01:46:39.987453 (+ 204us) tablet.cc:701] Row locks acquired
0807 01:46:39.987473 (+ 20us) write_op.cc:260] PREPARE: finished
0807 01:46:39.987842 (+ 369us) write_op.cc:270] Start()
0807 01:46:39.987954 (+ 112us) write_op.cc:276] Timestamp: P: 1786067199987785 usec, L: 0
0807 01:46:39.987975 (+ 21us) op_driver.cc:348] REPLICATION: starting
0807 01:46:39.988727 (+ 752us) log.cc:844] Serialized 531 byte log entry
0807 01:46:40.538906 (+550179us) op_driver.cc:464] REPLICATION: finished
0807 01:46:40.778004 (+239098us) write_op.cc:301] APPLY: starting
0807 01:46:40.783861 (+ 5857us) tablet.cc:1367] starting BulkCheckPresence
0807 01:46:40.817769 (+ 33908us) tablet.cc:1370] finished BulkCheckPresence
0807 01:46:40.844461 (+ 26692us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 01:46:40.863994 (+ 19533us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 01:46:40.864176 (+ 182us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0807 01:46:40.864223 (+ 47us) write_op.cc:312] APPLY: finished
0807 01:46:40.864490 (+ 267us) log.cc:844] Serialized 22 byte log entry
0807 01:46:40.864697 (+ 207us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:46:40.864797 (+ 100us) write_op.cc:454] Released schema lock
0807 01:46:40.864896 (+ 99us) write_op.cc:341] FINISH: Updating metrics
0807 01:46:40.865410 (+ 514us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:46:40.865487 (+ 77us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":202356,"apply.run_cpu_time_us":2759,"apply.run_wall_time_us":99992,"num_ops":1,"prepare.queue_time_us":139615,"prepare.run_cpu_time_us":3065,"prepare.run_wall_time_us":25321,"replication_time_us":538135,"thread_start_us":232590,"threads_started":2}]]}
I20260807 01:46:41.551326 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:41.569363 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:41.800625 28321 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:41.805970 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:41.470839 28354 raft_consensus.cc:697] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
W20260807 01:46:41.820323 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 301723us
I20260807 01:46:41.830823 27752 leader_election.cc:290] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:41.805178 27213 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:46:41.824435 28310 raft_consensus.cc:3060] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 01:46:41.824589 28233 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:41.821566 28209 raft_consensus.cc:493] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:41.830299 28367 raft_consensus.cc:3060] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:42.002391 27104 raft_consensus.cc:2393] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 3: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:41.840487 28370 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:41.997133 27331 raft_consensus.cc:3060] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:41.996701 27327 raft_consensus.cc:3060] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 3
I20260807 01:46:41.825016 28242 raft_consensus.cc:515] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:41.993927 28321 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.015072 28358 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:41.827656 28355 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:41.828240 28153 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.043330 28361 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:42.008471 28222 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:42.051360 28321 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.007265 28209 raft_consensus.cc:515] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.055507 28303 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.056071 28084 consensus_queue.cc:1048] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:42.013928 28345 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:42.026669 28206 consensus_queue.cc:237] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.066615 27229 raft_consensus.cc:2468] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:46:42.127514 28396 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:42.027181 28354 consensus_queue.cc:237] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.130656 28232 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:42.131130 28321 raft_consensus.cc:493] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:42.133625 26954 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46134 (request call id 125) took 2028 ms. Trace:
W20260807 01:46:42.134711 27036 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 01:46:42.130393 28242 leader_election.cc:290] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:42.067438 28071 raft_consensus.cc:493] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:42.134660 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:46:42.161470 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 165452us
I20260807 01:46:42.162665 27064 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:42.137370 28291 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.133687 28361 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.146129 26954 rpcz_store.cc:276] 0807 01:46:40.105436 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:40.105652 (+ 216us) service_pool.cc:224] Handling call
0807 01:46:42.133551 (+2027899us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 01:46:40.163422 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 01:46:40.236282 (+ 72860us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b5c0023ff8c. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d5b64a4f1 00007f7d5b64a3e1 00007f7d5b64512c 00007f7d58e88749 00007f7d58e73a86 00007f7d58e73738 00007f7d58e783d1 00007f7d58e78389 00007f7d58e78319 00007f7d58e782e1 00007f7d58e7750d
0807 01:46:40.236395 (+ 113us) write_op.cc:432] Acquiring schema lock in shared mode
0807 01:46:40.236435 (+ 40us) write_op.cc:435] Acquired schema lock
0807 01:46:40.236506 (+ 71us) tablet.cc:662] Decoding operations
0807 01:46:40.236989 (+ 483us) write_op.cc:620] Acquiring the partition lock for write op
0807 01:46:40.237058 (+ 69us) write_op.cc:641] Partition lock acquired for write op
0807 01:46:40.237119 (+ 61us) tablet.cc:685] Acquiring locks for 1 operations
0807 01:46:40.237435 (+ 316us) tablet.cc:701] Row locks acquired
0807 01:46:40.237449 (+ 14us) write_op.cc:260] PREPARE: finished
0807 01:46:40.237621 (+ 172us) write_op.cc:270] Start()
0807 01:46:40.237851 (+ 230us) write_op.cc:276] Timestamp: P: 1786067200237570 usec, L: 0
0807 01:46:40.237879 (+ 28us) op_driver.cc:348] REPLICATION: starting
0807 01:46:40.266813 (+ 28934us) log.cc:844] Serialized 531 byte log entry
0807 01:46:40.948091 (+681278us) op_driver.cc:464] REPLICATION: finished
0807 01:46:40.959060 (+ 10969us) write_op.cc:301] APPLY: starting
0807 01:46:40.959273 (+ 213us) tablet.cc:1367] starting BulkCheckPresence
0807 01:46:40.959422 (+ 149us) tablet.cc:1370] finished BulkCheckPresence
0807 01:46:40.959452 (+ 30us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 01:46:40.959707 (+ 255us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 01:46:40.959919 (+ 212us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0807 01:46:40.959977 (+ 58us) write_op.cc:312] APPLY: finished
0807 01:46:40.960225 (+ 248us) log.cc:844] Serialized 22 byte log entry
0807 01:46:40.960501 (+ 276us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:46:40.960609 (+ 108us) write_op.cc:454] Released schema lock
0807 01:46:40.960656 (+ 47us) write_op.cc:341] FINISH: Updating metrics
0807 01:46:40.961050 (+ 394us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:46:40.961082 (+ 32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":102216,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":3537,"apply.run_cpu_time_us":2157,"apply.run_wall_time_us":2159,"num_ops":1,"prepare.queue_time_us":335,"prepare.run_cpu_time_us":7802,"prepare.run_wall_time_us":156044,"replication_time_us":704847,"spinlock_wait_cycles":43145984}]]}
I20260807 01:46:42.078605 28222 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:42.159936 28306 consensus_meta.cc:287] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade: Time spent flushing consensus metadata: real 0.820s user 0.006s sys 0.010s
I20260807 01:46:42.070665 28358 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.138968 28345 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.154116 28232 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.156144 28321 raft_consensus.cc:515] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.154282 28071 raft_consensus.cc:515] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.155191 28396 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.184037 27109 raft_consensus.cc:2368] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3.
I20260807 01:46:42.180430 28306 raft_consensus.cc:2749] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3.
I20260807 01:46:42.180047 28394 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:42.196395 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:42.198534 27212 raft_consensus.cc:1275] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:42.200004 27103 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 459) took 1137 ms. Trace:
I20260807 01:46:42.200508 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:42.201049 28235 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:42.201175 28153 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.201508 27752 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:42.200114 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:42.202420 28402 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:42.187876 28367 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.180516 27102 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 468) took 1025 ms. Trace:
I20260807 01:46:42.222587 28404 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:42.202994 27229 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 3
I20260807 01:46:42.196239 28333 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:42.204748 27101 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 464) took 1090 ms. Trace:
I20260807 01:46:42.229202 27229 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:46:41.824127 27328 rpcz_store.cc:276] 0807 01:46:40.564858 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:40.564977 (+ 119us) service_pool.cc:224] Handling call
0807 01:46:40.669888 (+104911us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:46:41.057033 (+387145us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:46:41.057069 (+ 36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:46:41.058064 (+ 995us) log.cc:844] Serialized 26 byte log entry
0807 01:46:41.272285 (+214221us) raft_consensus.cc:1645] Marking committed up to 0
0807 01:46:41.272322 (+ 37us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:46:41.281303 (+ 8981us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:46:41.804853 (+523550us) raft_consensus.cc:1685] finished
0807 01:46:41.804875 (+ 22us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:46:41.805066 (+ 191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":50938,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":13720,"thread_start_us":555584,"threads_started":2,"wal-append.queue_time_us":210172}
I20260807 01:46:42.209589 27752 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.210826 28355 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.211318 28235 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.211783 28358 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:42.212076 27216 raft_consensus.cc:1275] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:42.213161 28345 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:42.213272 28232 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.214525 28222 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:42.214927 28071 leader_election.cc:290] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.215916 28209 leader_election.cc:290] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.217465 28242 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:42.217940 28221 raft_consensus.cc:493] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:42.201952 27230 raft_consensus.cc:2468] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:46:42.212019 27109 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41080 (request call id 466) took 1090 ms. Trace:
I20260807 01:46:42.209504 28291 raft_consensus.cc:2804] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:42.235807 27752 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.220206 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:42.221767 28347 raft_consensus.cc:493] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:42.223203 28404 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.237428 28355 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:42.260242 28367 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:42.255311 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:42.271090 28403 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:42.248520 28335 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.253261 28221 raft_consensus.cc:515] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.257050 28347 raft_consensus.cc:515] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.252174 28242 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.264043 28411 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:42.264930 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:42.230958 27101 rpcz_store.cc:276] 0807 01:46:41.114678 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:41.301946 (+187268us) service_pool.cc:224] Handling call
0807 01:46:42.203976 (+902030us) spinlock_profiling.cc:243] Waited 6.89 ms on lock 0x7b5800024c98. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e0521 00007f7d522e03d2 00007f7d5b24c219 00007f7d5a39f640 00007f7d5a467ebb 00007f7d5a467f41 00007f7d5a429dd0 00007f7d5a414739 00007f7d5a4141e2 00007f7d53959d34
0807 01:46:42.204199 (+ 223us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:46:42.204381 (+ 182us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:46:42.204419 (+ 38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:46:42.204460 (+ 41us) raft_consensus.cc:1645] Marking committed up to 1
0807 01:46:42.204495 (+ 35us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:46:42.204554 (+ 59us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:46:42.204674 (+ 120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15846656}
I20260807 01:46:42.219937 28394 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.268105 28361 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:42.222401 27102 rpcz_store.cc:276] 0807 01:46:41.154239 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:41.329701 (+175462us) service_pool.cc:224] Handling call
0807 01:46:41.345309 (+ 15608us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:46:41.345952 (+ 643us) raft_consensus.cc:1534] Early marking committed up to index 1
0807 01:46:41.603586 (+257634us) log.cc:844] Serialized 14 byte log entry
0807 01:46:42.154045 (+550459us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:46:42.157197 (+ 3152us) raft_consensus.cc:1645] Marking committed up to 1
0807 01:46:42.161210 (+ 4013us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:46:42.172422 (+ 11212us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:46:42.177570 (+ 5148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2342,"prepare.queue_time_us":472836,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":18692,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"thread_start_us":537618,"threads_started":2,"wal-append.queue_time_us":366057}
I20260807 01:46:42.263989 28410 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:42.221891 28402 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.265048 27216 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:46:42.245662 28333 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.261786 27105 raft_consensus.cc:2468] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:42.208782 27103 rpcz_store.cc:276] 0807 01:46:41.062091 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:41.067803 (+ 5712us) service_pool.cc:224] Handling call
0807 01:46:41.303492 (+235689us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b5800024c98. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e0521 00007f7d522e089e 000000000038d209 00007f7d5a39f940 00007f7d5a467ed4 00007f7d5a467f41 00007f7d5a429dd0 00007f7d5a414739 00007f7d5a4141e2 00007f7d53959d34
0807 01:46:42.199234 (+895742us) spinlock_profiling.cc:243] Waited 5.88 ms on lock 0x7b5800024c98. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e093c 00007f7d522e08a6 000000000038d209 00007f7d5a39f940 00007f7d5a467ed4 00007f7d5a467f41 00007f7d5a429dd0 00007f7d5a414739 00007f7d5a4141e2 00007f7d53959d34
0807 01:46:42.199381 (+ 147us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:46:42.199547 (+ 166us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:46:42.199574 (+ 27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:46:42.199704 (+ 130us) raft_consensus.cc:1645] Marking committed up to 1
0807 01:46:42.199747 (+ 43us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:46:42.199806 (+ 59us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:46:42.199919 (+ 113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19837952}
I20260807 01:46:42.261883 27327 raft_consensus.cc:2468] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:46:42.258637 28355 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.253038 27109 rpcz_store.cc:276] 0807 01:46:41.122525 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:41.193840 (+ 71315us) service_pool.cc:224] Handling call
0807 01:46:42.071359 (+877519us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b5800024c98. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e0521 00007f7d522e03d2 00007f7d5b24c219 00007f7d5a39f640 00007f7d5a467ebb 00007f7d5a467f41 00007f7d5a415eb0 00007f7d5a415794 00007f7d5395e834 00007f7d5395e7c2
0807 01:46:42.211938 (+140579us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6665472}
I20260807 01:46:42.298679 28221 leader_election.cc:290] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:42.278473 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:42.272115 28403 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.298516 28410 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.303007 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:42.309288 28396 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:42.300313 28291 raft_consensus.cc:697] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:42.310550 28238 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.316881 28233 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.318368 28351 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:42.321255 28400 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:42.323709 28321 leader_election.cc:290] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.322326 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:42.342988 27226 raft_consensus.cc:2393] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 3: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:42.329668 28235 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:46:42.338567 28369 log.cc:927] Time spent T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.991s user 0.005s sys 0.006s
I20260807 01:46:42.327797 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:42.327145 27099 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:46:42.302107 28411 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.340687 28412 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
W20260807 01:46:42.348609 28407 log.cc:927] Time spent T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.097s user 0.002s sys 0.000s
I20260807 01:46:42.355361 28404 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.361440 27222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57662 (request call id 338) took 1885 ms. Trace:
I20260807 01:46:42.364846 28242 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:42.372071 28291 consensus_queue.cc:237] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.376473 27338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59490 (request call id 399) took 1058 ms. Trace:
I20260807 01:46:42.378268 27338 rpcz_store.cc:276] 0807 01:46:41.318343 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:41.318755 (+ 412us) service_pool.cc:224] Handling call
0807 01:46:41.319074 (+ 319us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:46:41.330581 (+ 11507us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:46:41.330610 (+ 29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:46:42.165181 (+834571us) spinlock_profiling.cc:243] Waited 6.87 ms on lock 0x7b2c00110d18. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d53d19c71 00007f7d53d295d2 00007f7d55459de1 00007f7d5541e81a 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22
0807 01:46:42.165598 (+ 417us) log.cc:844] Serialized 26 byte log entry
0807 01:46:42.184156 (+ 18558us) raft_consensus.cc:1645] Marking committed up to 0
0807 01:46:42.184190 (+ 34us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:46:42.184473 (+ 283us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:46:42.376024 (+191551us) raft_consensus.cc:1685] finished
0807 01:46:42.376049 (+ 25us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:46:42.376407 (+ 358us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11069,"raft.queue_time_us":11261,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":8854,"spinlock_wait_cycles":15797120,"thread_start_us":18124,"threads_started":1,"wal-append.queue_time_us":55856}
I20260807 01:46:42.380569 28347 leader_election.cc:290] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.340394 28351 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:42.386626 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 129140us
I20260807 01:46:42.387782 27064 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:42.387784 28403 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.388686 28411 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.389099 28403 raft_consensus.cc:2804] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:42.370857 27102 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:42.351907 28400 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.392638 27064 leader_election.cc:304] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:42.383694 28335 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:42.396890 28394 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:46:42.401320 27063 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.510s (SENT) [suppressed 64 similar messages]
I20260807 01:46:42.373665 27222 rpcz_store.cc:276] 0807 01:46:40.475623 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:40.477799 (+ 2176us) service_pool.cc:224] Handling call
0807 01:46:40.687293 (+209494us) spinlock_profiling.cc:243] Waited 1.86 ms on lock 0x7b5800135398. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e0521 00007f7d522e03d2 00007f7d5b24c219 00007f7d5a39f640 00007f7d5a467ebb 00007f7d5a467f41 00007f7d5a429dd0 00007f7d5a414739 00007f7d5a4141e2 00007f7d53959d34
0807 01:46:40.687540 (+ 247us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:46:40.708645 (+ 21105us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:46:40.710886 (+ 2241us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:46:40.741707 (+ 30821us) log.cc:844] Serialized 26 byte log entry
0807 01:46:41.187965 (+446258us) raft_consensus.cc:1645] Marking committed up to 1
0807 01:46:41.197064 (+ 9099us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:46:41.197349 (+ 285us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:46:42.230829 (+1033480us) spinlock_profiling.cc:243] Waited 1.86 ms on lock 0x7b440002250c. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d5223e855 00007f7d5223f272 00007f7d54fc93bb 00007f7d54fc4c9f 00007f7d54fb8a5e 00007f7d54fb7da6 00007f7d54fb79c8 00007f7d54fc1411 00007f7d54fc13c9 00007f7d54fc1359
0807 01:46:42.361126 (+130297us) raft_consensus.cc:1685] finished
0807 01:46:42.361159 (+ 33us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:46:42.361365 (+ 206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6870,"raft.queue_time_us":39201,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":8836,"spinlock_wait_cycles":8565760,"thread_start_us":375011,"threads_started":1,"wal-append.queue_time_us":504505}
I20260807 01:46:42.393121 28403 raft_consensus.cc:697] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:42.398615 28370 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.410173 27104 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 287) took 1307 ms. Trace:
I20260807 01:46:42.371789 28412 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.410557 27104 rpcz_store.cc:276] 0807 01:46:41.101422 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:41.111183 (+ 9761us) service_pool.cc:224] Handling call
0807 01:46:42.409047 (+1297864us) spinlock_profiling.cc:243] Waited 888 ms on lock 0x7b5400171bd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:46:42.409197 (+ 150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2042577664}
I20260807 01:46:42.392009 28411 consensus_queue.cc:1048] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260807 01:46:42.402729 28347 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:42.404979 28355 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.407691 28404 raft_consensus.cc:2804] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:42.394691 28321 consensus_queue.cc:1048] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260807 01:46:42.420552 28412 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.421041 27331 raft_consensus.cc:2468] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:46:42.408581 28410 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.426540 27110 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41080 (request call id 465) took 1307 ms. Trace:
I20260807 01:46:42.423527 27336 raft_consensus.cc:2393] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 3: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:42.426898 27110 rpcz_store.cc:276] 0807 01:46:41.118929 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:41.287609 (+168680us) service_pool.cc:224] Handling call
0807 01:46:42.423204 (+1135595us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b540024ead4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:46:42.426475 (+ 3271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":391390976}
I20260807 01:46:42.416430 28333 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.427135 28233 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.434356 28404 raft_consensus.cc:697] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:42.435791 28404 consensus_queue.cc:237] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.479641 28403 consensus_queue.cc:237] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.481498 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:42.485643 28414 log.cc:927] Time spent T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.055s user 0.000s sys 0.001s
I20260807 01:46:42.486944 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:42.431293 28402 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:42.487782 28415 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:42.487694 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:42.488425 28351 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.480338 28400 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:42.488474 28410 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:42.422354 28347 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.515806 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:42.516204 28370 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:42.499727 28354 raft_consensus.cc:493] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:42.516728 27227 raft_consensus.cc:2468] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:42.516999 28410 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:42.525763 28310 consensus_meta.cc:287] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.522s user 0.012s sys 0.000s
I20260807 01:46:42.526676 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:42.527405 27227 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:42.509904 27213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57654 (request call id 301) took 1203 ms. Trace:
I20260807 01:46:42.532765 28310 raft_consensus.cc:2749] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 01:46:42.533814 28347 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.536777 27063 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:42.544869 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:42.522454 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:42.521517 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:42.518539 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:42.549626 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:42.526229 28354 raft_consensus.cc:515] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.512454 28415 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.564927 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:42.546363 28400 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:42.565816 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:42.571960 28400 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.567451 27108 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:42.548617 28402 raft_consensus.cc:493] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:42.568377 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:42.571744 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:42.580135 28328 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:42.547523 28422 raft_consensus.cc:2749] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:42.595528 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:42.606889 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:42.607316 28321 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:42.602234 28394 raft_consensus.cc:493] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:42.610772 27108 raft_consensus.cc:3060] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:42.610921 27338 raft_consensus.cc:2468] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:42.619576 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:42.562453 27212 raft_consensus.cc:2468] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:42.566461 27222 raft_consensus.cc:2468] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:42.602749 28335 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:42.602608 28402 raft_consensus.cc:515] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.629232 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:42.615521 27102 raft_consensus.cc:2468] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:42.635294 27108 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:46:42.635599 27336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33976 (request call id 332) took 1114 ms. Trace:
I20260807 01:46:42.624019 28306 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:42.635933 27336 rpcz_store.cc:276] 0807 01:46:41.518067 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:41.564191 (+ 46124us) service_pool.cc:224] Handling call
0807 01:46:42.632711 (+1068520us) spinlock_profiling.cc:243] Waited 589 ms on lock 0x7b540027bfd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:46:42.632822 (+ 111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1354435456}
I20260807 01:46:42.620627 28321 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.643056 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:42.634069 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:42.634953 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:42.622341 28394 raft_consensus.cc:515] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.651067 27103 raft_consensus.cc:3060] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:42.624199 28328 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.642514 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:42.647773 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:42.636336 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:42.637069 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:42.640473 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:42.649080 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:42.654043 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:42.633581 28335 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.655892 28415 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.656520 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:42.657052 28335 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:42.644908 28410 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.657258 27336 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:46:42.658895 28394 leader_election.cc:290] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:42.649411 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:42.661260 27212 raft_consensus.cc:3060] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:42.663371 28321 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.641322 28306 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.539249 27213 rpcz_store.cc:276] 0807 01:46:41.306388 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:41.311496 (+ 5108us) service_pool.cc:224] Handling call
0807 01:46:42.506731 (+1195235us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b54001de054. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:46:42.509795 (+ 3064us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77628288}
I20260807 01:46:42.667701 28394 raft_consensus.cc:493] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:42.669548 27287 leader_election.cc:400] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3.
I20260807 01:46:42.669790 27287 leader_election.cc:403] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260807 01:46:42.666867 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:42.675172 27287 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:42.653925 27099 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:42.675688 28335 raft_consensus.cc:2749] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3.
I20260807 01:46:42.670940 28394 raft_consensus.cc:515] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.684437 28394 leader_election.cc:290] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:42.690621 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:42.691627 27101 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:46:42.668247 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:42.693452 27287 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:42.693869 28242 raft_consensus.cc:2749] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:42.694554 28370 raft_consensus.cc:2804] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:42.694842 28242 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:42.695153 28370 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:42.695767 28370 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:42.695648 28242 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.698777 28396 raft_consensus.cc:493] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:42.704133 27287 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:42.704351 27064 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:42.671213 28328 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:42.713301 28321 raft_consensus.cc:2804] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:46:42.721833 28321 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:42.722291 28321 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:42.722833 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:42.724674 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:42.726409 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:42.700695 28396 raft_consensus.cc:515] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.728214 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:42.728761 27218 raft_consensus.cc:2468] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:42.719453 27170 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:42.701062 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:42.710805 28221 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:42.720661 28410 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:42.738495 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:42.718997 28242 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:42.738680 28410 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.742758 27103 raft_consensus.cc:2468] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:46:42.715440 27330 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:46:42.730516 27064 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:42.732508 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:42.735008 28396 leader_election.cc:290] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:42.748716 27064 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:42.738401 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:42.759158 27064 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:42.762655 27064 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:42.770422 28290 raft_consensus.cc:2804] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:42.771047 28290 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:42.771584 28290 raft_consensus.cc:3060] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:42.773209 27212 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:46:42.766453 28405 raft_consensus.cc:2804] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:42.776202 28405 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:42.776587 28405 raft_consensus.cc:3060] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:42.753057 28291 raft_consensus.cc:2804] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:42.782964 28291 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:42.783391 28291 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:42.786222 27216 raft_consensus.cc:3060] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:42.787087 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:42.790392 28095 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:42.791337 28095 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.755322 28155 raft_consensus.cc:2804] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:42.751183 28410 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.793839 28155 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:42.764252 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:42.796824 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:42.755508 27102 raft_consensus.cc:2468] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:42.797832 28347 raft_consensus.cc:2764] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260807 01:46:42.794552 28155 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:42.810562 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:42.846482 28403 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:42.847512 28403 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } }
I20260807 01:46:42.853025 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:42.854336 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:42.853742 27108 raft_consensus.cc:2468] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:42.819556 28084 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:42.839614 28394 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:42.838505 28232 raft_consensus.cc:493] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:42.838836 28422 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:42.841926 28238 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:42.840284 28355 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:42.853737 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:42.880275 28095 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.862953 28306 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:42.856385 28410 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:42.900920 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:42.893854 27218 raft_consensus.cc:2393] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 3: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:42.935101 28290 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.875815 28084 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.908644 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:42.913772 28370 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.890408 28422 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.914811 28402 leader_election.cc:290] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:42.916066 28354 leader_election.cc:290] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.917325 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:42.919145 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:42.921355 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:42.895506 28355 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.922906 27326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33976 (request call id 330) took 1405 ms. Trace:
I20260807 01:46:42.892342 28232 raft_consensus.cc:515] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.926098 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:42.926472 27214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57662 (request call id 359) took 1617 ms. Trace:
I20260807 01:46:42.928068 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:42.929893 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:42.929690 28410 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.883008 28394 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.930750 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:42.872843 28206 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:42.893582 28238 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:42.948549 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:42.949903 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:42.951247 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:42.942914 28403 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:42.957996 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:42.973623 27287 leader_election.cc:304] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:43.000622 27063 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
W20260807 01:46:43.027184 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 53640.2us
I20260807 01:46:43.028436 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.028628 27287 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:43.006596 27230 raft_consensus.cc:2468] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:46:43.024822 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.007491 27338 raft_consensus.cc:2376] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3. Re-sending same reply.
W20260807 01:46:43.006053 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 109807us
I20260807 01:46:43.026245 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.000774 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:43.029888 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:46:43.046228 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 90060us
I20260807 01:46:43.046998 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:43.048130 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:43.030512 27063 leader_election.cc:304] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.050009 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.051292 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.052071 27336 raft_consensus.cc:2410] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260807 01:46:43.052925 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.055719 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.042021 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.043167 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.054047 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.028934 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:43.072893 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:43.037334 28403 raft_consensus.cc:2749] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.040031 28306 raft_consensus.cc:2804] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:43.035740 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.086199 28238 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.059302 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.086912 28232 leader_election.cc:290] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.087895 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.088178 28370 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:43.087742 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:43.076890 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:43.089262 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.089861 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:43.034617 28206 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.079115 27101 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:46:43.090762 27100 raft_consensus.cc:2393] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 3: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:43.009781 27214 rpcz_store.cc:276] 0807 01:46:41.308677 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:41.309299 (+ 622us) service_pool.cc:224] Handling call
0807 01:46:42.179693 (+870394us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b5800135398. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e0521 00007f7d522e089e 000000000038d209 00007f7d5a39f940 00007f7d5a467ed4 00007f7d5a467f41 00007f7d5a429dd0 00007f7d5a414739 00007f7d5a4141e2 00007f7d53959d34
0807 01:46:42.184744 (+ 5051us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:46:42.231175 (+ 46431us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:46:42.233786 (+ 2611us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:46:42.238977 (+ 5191us) raft_consensus.cc:1645] Marking committed up to 1
0807 01:46:42.244593 (+ 5616us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:46:42.926062 (+681469us) spinlock_profiling.cc:243] Waited 30.5 ms on lock 0x7b54002ee0d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:46:42.926205 (+ 143us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:46:42.926401 (+ 196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100495616}
I20260807 01:46:43.092070 28206 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.094048 28221 raft_consensus.cc:2749] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260807 01:46:43.095924 28440 log.cc:927] Time spent T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.063s user 0.000s sys 0.003s
I20260807 01:46:43.095813 28405 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.097833 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.065563 28321 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.090334 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:43.102875 28291 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.074002 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.059088 27222 raft_consensus.cc:2393] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 3: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:43.044917 28396 raft_consensus.cc:2804] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:46:43.049544 28155 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.075054 28211 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:43.034942 27331 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:43.075296 28404 raft_consensus.cc:2749] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.080276 28306 raft_consensus.cc:493] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 01:46:43.078910 27330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33976 (request call id 313) took 1971 ms (client timeout 1538 ms). Trace:
I20260807 01:46:43.083278 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.084014 28410 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.084635 28290 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.085127 28084 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.085561 28355 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.068557 28394 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:43.105255 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.009606 27326 rpcz_store.cc:276] 0807 01:46:41.517548 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:41.989617 (+472069us) service_pool.cc:224] Handling call
0807 01:46:42.010875 (+ 21258us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:46:42.198213 (+187338us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:46:42.198243 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:46:42.198755 (+ 512us) log.cc:844] Serialized 26 byte log entry
0807 01:46:42.245208 (+ 46453us) raft_consensus.cc:1645] Marking committed up to 1
0807 01:46:42.246337 (+ 1129us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:46:42.922441 (+676104us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b5400165654. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:46:42.922493 (+ 52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:46:42.922544 (+ 51us) raft_consensus.cc:1685] finished
0807 01:46:42.922558 (+ 14us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:46:42.922838 (+ 280us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1182,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":117,"spinlock_wait_cycles":383618944,"thread_start_us":32543,"threads_started":1,"wal-append.queue_time_us":42710}
I20260807 01:46:43.120505 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.126988 28291 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.127439 28211 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.127560 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.192690 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.129372 28306 raft_consensus.cc:3060] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:43.131096 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:46:43.129523 27330 rpcz_store.cc:269] 0807 01:46:41.107094 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:41.116762 (+ 9668us) service_pool.cc:224] Handling call
0807 01:46:42.264785 (+1148023us) spinlock_profiling.cc:243] Waited 2.15 ms on lock 0x7b5800003698. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e0521 00007f7d522e089e 000000000038d209 00007f7d5a39f940 00007f7d5a467ed4 00007f7d5a467f41 00007f7d5a415eb0 00007f7d5a415794 00007f7d5395e834 00007f7d5395e7c2
0807 01:46:43.071841 (+807056us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b5400164c54. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:46:43.076451 (+ 4610us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":723572736}
I20260807 01:46:43.135900 28304 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:43.202963 28304 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.157234 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:46:43.207492 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 160145us
I20260807 01:46:43.160656 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:43.163331 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.144306 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.211572 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:43.160578 28396 raft_consensus.cc:697] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:43.213012 28396 consensus_queue.cc:237] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.215194 28211 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:43.133770 28155 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.159942 27103 raft_consensus.cc:2393] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 3: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:43.143812 27216 raft_consensus.cc:2468] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:46:43.218993 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:43.219969 27216 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:43.149389 27327 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:46:43.223922 28304 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.227916 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.208245 27170 leader_election.cc:304] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:43.180751 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.112066 27337 raft_consensus.cc:1275] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:43.229799 27170 leader_election.cc:304] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.166280 27064 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:43.239990 27287 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:43.240653 27170 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
W20260807 01:46:43.244410 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 78238.2us
I20260807 01:46:43.136806 28405 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.247515 28144 raft_consensus.cc:2804] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:43.248510 28144 raft_consensus.cc:493] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:43.245401 28155 raft_consensus.cc:2804] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:43.249977 28155 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:43.245946 28444 raft_consensus.cc:2749] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.250752 28155 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:43.249179 28144 raft_consensus.cc:3060] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
W20260807 01:46:43.254144 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 204579us
I20260807 01:46:43.257640 27287 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:43.260851 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.261827 27338 raft_consensus.cc:2393] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 3: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:43.143661 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.155714 28400 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:43.288107 27287 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:43.237684 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:43.303491 27105 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:46:43.128969 27331 raft_consensus.cc:1275] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:43.179010 28422 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.172968 28321 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.313771 27287 leader_election.cc:304] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:43.323448 27287 leader_election.cc:304] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:43.325536 27287 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:43.258999 28232 raft_consensus.cc:2804] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:43.292009 28402 raft_consensus.cc:2804] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:43.347138 28402 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:43.347589 28402 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:43.321269 28422 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:43.348748 28422 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.349726 27222 raft_consensus.cc:1275] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:43.278324 28335 raft_consensus.cc:2749] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:46:43.330274 28370 raft_consensus.cc:2804] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:43.350829 28370 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:43.335541 28211 raft_consensus.cc:2749] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.351006 28422 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.352404 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.328465 28221 raft_consensus.cc:2749] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.325270 28242 raft_consensus.cc:2749] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.318589 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.339684 27229 raft_consensus.cc:1275] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:43.293985 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.356247 27218 raft_consensus.cc:1275] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:43.356745 28370 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:43.314260 28400 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:43.346570 28232 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:43.356455 27229 raft_consensus.cc:1275] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:43.358449 27230 raft_consensus.cc:1275] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:43.358654 28422 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:43.359229 28422 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.362360 28422 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.362686 28400 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:43.362833 27036 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 01:46:43.328933 27287 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
W20260807 01:46:43.365092 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 89075.6us
I20260807 01:46:43.366431 27287 leader_election.cc:304] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:43.362269 28232 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:43.373431 28155 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.380512 27172 leader_election.cc:400] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3.
I20260807 01:46:43.380889 27172 leader_election.cc:403] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260807 01:46:43.345973 27216 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:46:43.381927 27172 leader_election.cc:400] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3.
I20260807 01:46:43.385366 27172 leader_election.cc:403] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260807 01:46:43.386032 27287 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:43.377895 28400 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:43.382531 28441 raft_consensus.cc:2749] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3.
I20260807 01:46:43.384002 28370 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.376211 28394 raft_consensus.cc:2804] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:43.376313 28144 raft_consensus.cc:515] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.389654 27287 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:43.394062 28396 raft_consensus.cc:2804] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:46:43.394798 28396 raft_consensus.cc:697] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:43.387287 28242 raft_consensus.cc:2749] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.392477 28394 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:43.397078 28394 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:43.397254 28444 raft_consensus.cc:2749] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3.
I20260807 01:46:43.397936 27172 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.398798 28441 consensus_queue.cc:1048] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:43.400178 28441 raft_consensus.cc:2749] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.399163 27172 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.402607 28238 raft_consensus.cc:2804] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:43.402994 28238 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:43.403358 28238 raft_consensus.cc:3060] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:43.403685 27172 leader_election.cc:304] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.404454 28438 raft_consensus.cc:2749] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.392989 28400 raft_consensus.cc:2749] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:46:43.408010 27172 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.410724 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:43.346002 28306 raft_consensus.cc:515] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:43.417222 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 141203us
I20260807 01:46:43.418025 27170 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.419823 28403 consensus_queue.cc:1048] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:43.422421 27287 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:43.422922 28321 consensus_queue.cc:1048] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:43.423243 28333 raft_consensus.cc:2749] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.428364 28400 raft_consensus.cc:2804] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:43.428977 28400 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:43.429463 28400 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:43.417519 28233 raft_consensus.cc:2804] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:43.430636 27170 leader_election.cc:304] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.423945 27287 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:43.434645 28438 raft_consensus.cc:2749] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.438004 28155 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.438673 28321 consensus_queue.cc:1048] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:43.430819 28233 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:43.441177 28233 raft_consensus.cc:3060] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:43.411423 26949 catalog_manager.cc:5697] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:43.444204 28144 leader_election.cc:290] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.448005 28403 consensus_queue.cc:1048] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:43.455672 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:43.467869 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.468139 28306 leader_election.cc:290] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:43.468308 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:43.470468 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.470896 28370 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:43.455349 28232 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.472409 27064 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
W20260807 01:46:43.475359 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 51472.7us
I20260807 01:46:43.477536 27063 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.479350 27063 leader_election.cc:304] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.395788 28396 consensus_queue.cc:237] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.488165 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:43.483067 27063 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.484786 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:43.488811 27108 raft_consensus.cc:3060] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:43.486515 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:43.491560 27063 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.481433 28084 raft_consensus.cc:2749] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.495548 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:43.498695 27213 raft_consensus.cc:2393] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 3: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:43.498909 28458 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:43.499392 28458 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.501361 28458 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.501806 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.502053 28422 raft_consensus.cc:2749] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.494656 28304 raft_consensus.cc:2749] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.480234 28155 consensus_queue.cc:1048] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260807 01:46:43.490685 28403 raft_consensus.cc:2804] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:46:43.508091 28403 raft_consensus.cc:697] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:43.508424 28400 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.509037 28403 consensus_queue.cc:237] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.494504 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:43.510676 28402 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.511142 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:43.511938 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:43.503639 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.503543 28232 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.501250 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:43.512732 27101 raft_consensus.cc:2393] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 3: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:43.491151 28321 raft_consensus.cc:2749] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.482810 28306 raft_consensus.cc:2804] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:43.522804 27108 raft_consensus.cc:2468] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:46:43.502089 28233 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.524024 27063 leader_election.cc:304] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.537088 27063 leader_election.cc:304] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.538540 27063 leader_election.cc:304] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.539762 27063 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.541380 28422 raft_consensus.cc:2749] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.543381 27105 raft_consensus.cc:1240] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:43.523584 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:43.530068 28221 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:43.544521 27212 raft_consensus.cc:2393] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 3: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:43.544265 28221 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.540680 28409 raft_consensus.cc:2749] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.527002 28306 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:43.546455 28304 raft_consensus.cc:2749] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.546468 28306 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:43.547768 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.548386 28400 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:43.551872 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.549549 28238 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.553089 27099 raft_consensus.cc:1240] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:43.488214 28394 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.555193 27226 raft_consensus.cc:1240] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:43.560527 28394 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:43.516016 28402 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:43.563444 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:43.564337 27229 raft_consensus.cc:1240] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:43.564184 27170 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.569051 27170 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.569406 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:43.570242 27226 raft_consensus.cc:2393] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 2: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:43.571502 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:43.573308 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:43.575439 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
W20260807 01:46:43.575723 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 46089.3us
I20260807 01:46:43.576709 27172 leader_election.cc:304] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:43.575474 27170 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.561244 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.571027 28144 raft_consensus.cc:2749] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.583988 28221 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:43.584257 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:43.585129 27109 raft_consensus.cc:2393] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 2: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:46:43.561558 28238 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.583081 27063 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:43.599360 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.599579 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.599885 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.600265 27336 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:43.604766 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:43.607059 27287 leader_election.cc:304] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:43.608728 27338 raft_consensus.cc:1275] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:43.610373 27333 raft_consensus.cc:1275] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:43.580907 28444 raft_consensus.cc:2749] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.610924 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:43.582782 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.592389 28235 raft_consensus.cc:2804] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:46:43.614946 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:43.614925 27287 leader_election.cc:304] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:43.580268 28291 consensus_queue.cc:1048] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260807 01:46:43.596149 28144 raft_consensus.cc:2749] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:46:43.601054 28221 raft_consensus.cc:493] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:43.624327 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:43.614583 28321 raft_consensus.cc:2804] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:43.623211 28235 raft_consensus.cc:697] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:43.626659 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.612165 28155 raft_consensus.cc:2749] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.629999 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.633934 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:43.638399 27337 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:43.636293 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:43.610127 28242 raft_consensus.cc:2749] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.639632 27336 raft_consensus.cc:2393] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 3: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:43.624828 28444 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:43.642473 28211 raft_consensus.cc:2749] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.642714 28444 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.644127 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:43.645634 28444 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.645850 27328 raft_consensus.cc:1275] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:43.599864 28403 consensus_queue.cc:1048] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260807 01:46:43.648429 27328 raft_consensus.cc:1275] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:43.645426 28306 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.650301 28306 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:43.652647 28233 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.658672 27287 leader_election.cc:304] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:43.660133 27287 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:43.663932 27287 leader_election.cc:304] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:43.664242 27337 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:46:43.621106 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.630091 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.667579 27287 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:43.669349 27287 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:43.670990 27333 raft_consensus.cc:1275] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:43.670003 28242 raft_consensus.cc:2749] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.663064 28306 raft_consensus.cc:2749] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.664758 28394 raft_consensus.cc:2749] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.678368 27287 leader_election.cc:304] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:43.663801 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.666749 28321 raft_consensus.cc:697] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:43.676983 28211 raft_consensus.cc:2749] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.674964 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.669905 28396 raft_consensus.cc:2749] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.682528 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:43.682694 28235 consensus_queue.cc:237] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.640486 28221 raft_consensus.cc:515] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.684309 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:43.684178 28306 raft_consensus.cc:2749] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.693876 28405 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:43.706979 27287 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:43.706539 28422 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:43.715747 28422 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.710400 28405 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.717777 28422 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.718920 28405 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.705224 28290 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:43.723757 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.724334 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.706375 28351 raft_consensus.cc:493] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:43.714721 28242 raft_consensus.cc:2749] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:46:43.729889 28290 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.730223 28351 raft_consensus.cc:515] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.725661 28321 consensus_queue.cc:237] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.753625 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.742535 28335 consensus_queue.cc:1059] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260807 01:46:43.770762 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.766125 28221 leader_election.cc:290] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:43.760167 28370 consensus_queue.cc:1059] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260807 01:46:43.775291 28290 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.776736 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:43.777802 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:43.779287 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:43.777896 27336 raft_consensus.cc:1240] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:43.782589 27333 raft_consensus.cc:2393] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 3: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:43.784641 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.786700 27331 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:46:43.788635 27338 raft_consensus.cc:1240] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:43.770650 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:43.801090 28335 raft_consensus.cc:3055] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:43.776356 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:46:43.819443 28467 log.cc:927] Time spent T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.154s user 0.000s sys 0.005s
I20260807 01:46:43.801563 28335 raft_consensus.cc:740] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:43.801852 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:43.828133 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.830565 27063 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.832145 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.832640 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:43.830178 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:43.813931 27325 raft_consensus.cc:2468] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:46:43.834023 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:43.837585 27063 leader_election.cc:304] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.839529 27110 raft_consensus.cc:1240] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:43.842667 27104 raft_consensus.cc:1240] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:43.833480 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:43.847945 27063 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:43.848632 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:43.849280 27229 raft_consensus.cc:1275] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260807 01:46:43.850067 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:43.851063 27212 raft_consensus.cc:2368] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 3. Current term is 4.
I20260807 01:46:43.852065 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:46:43.852505 28479 log.cc:927] Time spent T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.054s user 0.000s sys 0.000s
I20260807 01:46:43.853705 28351 leader_election.cc:290] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:43.854300 28351 raft_consensus.cc:2804] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:43.854650 28351 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:43.855435 27063 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.847936 27170 leader_election.cc:304] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.855024 28351 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:43.856041 28403 raft_consensus.cc:2749] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.873032 28501 raft_consensus.cc:2749] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.877074 27170 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.846241 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:43.834980 28335 consensus_queue.cc:260] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.877233 28304 consensus_queue.cc:1048] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:43.840817 28491 raft_consensus.cc:2749] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.888890 27287 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:43.875833 28144 raft_consensus.cc:2749] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:46:43.893142 27287 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:43.883358 28238 raft_consensus.cc:2749] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.896358 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:43.896853 28484 raft_consensus.cc:2749] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:46:43.897266 27326 raft_consensus.cc:2393] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 2: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:43.897825 28221 raft_consensus.cc:2749] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:43.872875 28502 consensus_queue.cc:1048] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260807 01:46:43.900478 28394 raft_consensus.cc:3055] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:43.900954 28394 raft_consensus.cc:740] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:43.901860 28394 consensus_queue.cc:260] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:43.904204 28394 raft_consensus.cc:3060] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:43.908357 28335 raft_consensus.cc:3060] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:43.909977 28351 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:46:43.872555 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 104410us
W20260807 01:46:43.959937 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 86491.1us
I20260807 01:46:43.870410 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:44.018576 27063 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:43.895396 27287 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:44.031024 28306 raft_consensus.cc:2749] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:46:44.032037 28409 raft_consensus.cc:2749] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260807 01:46:44.033644 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 138191us
I20260807 01:46:44.035197 27063 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:44.037972 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:44.038432 28505 raft_consensus.cc:2749] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:46:44.041726 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:44.042825 27063 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:44.043946 28505 raft_consensus.cc:2749] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:46:44.045300 27063 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:44.046485 28306 consensus_queue.cc:1059] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:44.050168 28402 raft_consensus.cc:3055] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:44.050786 28402 raft_consensus.cc:740] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:44.050411 28211 consensus_queue.cc:1059] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:44.052588 28402 consensus_queue.cc:260] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:44.058092 28211 raft_consensus.cc:3055] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:44.058658 28211 raft_consensus.cc:740] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:44.059474 28402 raft_consensus.cc:3060] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:44.059753 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:44.060151 28211 consensus_queue.cc:260] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:44.061817 28211 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:44.070515 28304 raft_consensus.cc:493] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:44.071475 28304 raft_consensus.cc:515] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:44.078636 28306 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:44.078982 27287 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:44.079561 28306 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:44.087162 28221 raft_consensus.cc:2749] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:46:44.090834 28409 raft_consensus.cc:2749] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:46:44.093055 28306 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:44.073933 28304 leader_election.cc:290] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:44.101446 28351 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:44.110126 27287 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:44.111361 28335 raft_consensus.cc:2749] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:46:44.111976 27287 leader_election.cc:304] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:44.112948 28335 raft_consensus.cc:2749] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260807 01:46:44.134287 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 89320.6us
I20260807 01:46:44.135890 28351 consensus_queue.cc:1048] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:44.138062 27063 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:44.146543 28351 raft_consensus.cc:2749] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:46:44.148561 26954 catalog_manager.cc:5697] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:44.151858 27100 raft_consensus.cc:1240] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3. Ops: []
I20260807 01:46:44.154021 26950 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46152 (request call id 117) took 1643 ms. Trace:
I20260807 01:46:44.155431 26954 catalog_manager.cc:5697] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:44.157205 28351 consensus_queue.cc:1048] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:44.138176 26949 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46138 (request call id 122) took 1894 ms. Trace:
I20260807 01:46:44.158646 26954 catalog_manager.cc:5697] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:44.160408 26949 rpcz_store.cc:276] 0807 01:46:42.243651 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:42.245408 (+ 1757us) service_pool.cc:224] Handling call
0807 01:46:44.137849 (+1892441us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 01:46:43.677700 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 01:46:43.682928 (+ 5228us) write_op.cc:432] Acquiring schema lock in shared mode
0807 01:46:43.682963 (+ 35us) write_op.cc:435] Acquired schema lock
0807 01:46:43.682981 (+ 18us) tablet.cc:662] Decoding operations
0807 01:46:43.683362 (+ 381us) write_op.cc:620] Acquiring the partition lock for write op
0807 01:46:43.683436 (+ 74us) write_op.cc:641] Partition lock acquired for write op
0807 01:46:43.683470 (+ 34us) tablet.cc:685] Acquiring locks for 1 operations
0807 01:46:43.683659 (+ 189us) tablet.cc:701] Row locks acquired
0807 01:46:43.683676 (+ 17us) write_op.cc:260] PREPARE: finished
0807 01:46:43.683819 (+ 143us) write_op.cc:270] Start()
0807 01:46:43.683920 (+ 101us) write_op.cc:276] Timestamp: P: 1786067203683778 usec, L: 0
0807 01:46:43.683939 (+ 19us) op_driver.cc:348] REPLICATION: starting
0807 01:46:43.731722 (+ 47783us) log.cc:844] Serialized 531 byte log entry
0807 01:46:43.919150 (+187428us) op_driver.cc:464] REPLICATION: finished
0807 01:46:43.920402 (+ 1252us) write_op.cc:301] APPLY: starting
0807 01:46:44.076082 (+155680us) tablet.cc:1367] starting BulkCheckPresence
0807 01:46:44.076318 (+ 236us) tablet.cc:1370] finished BulkCheckPresence
0807 01:46:44.097598 (+ 21280us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 01:46:44.098040 (+ 442us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 01:46:44.099496 (+ 1456us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0807 01:46:44.099572 (+ 76us) write_op.cc:312] APPLY: finished
0807 01:46:44.119239 (+ 19667us) log.cc:844] Serialized 22 byte log entry
0807 01:46:44.119587 (+ 348us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:46:44.119710 (+ 123us) write_op.cc:454] Released schema lock
0807 01:46:44.119773 (+ 63us) write_op.cc:341] FINISH: Updating metrics
0807 01:46:44.138625 (+ 18852us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:46:44.138659 (+ 34us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":14,"child_traces":[["op",{"apply.queue_time_us":1021,"apply.run_cpu_time_us":5237,"apply.run_wall_time_us":218097,"num_ops":1,"prepare.queue_time_us":117536,"prepare.run_cpu_time_us":17220,"prepare.run_wall_time_us":138436,"raft.queue_time_us":83387,"raft.run_cpu_time_us":1452,"raft.run_wall_time_us":1452,"replication_time_us":234964,"thread_start_us":188454,"threads_started":4,"wal-append.queue_time_us":97311}]]}
I20260807 01:46:44.167588 28402 consensus_queue.cc:1059] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:44.170964 28394 consensus_queue.cc:1059] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:44.156944 26950 rpcz_store.cc:276] 0807 01:46:42.513915 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:46:42.515573 (+ 1658us) service_pool.cc:224] Handling call
0807 01:46:44.153823 (+1638250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":202010,"reported_tablets":6}
I20260807 01:46:44.171483 27063 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:44.172073 28211 raft_consensus.cc:3055] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:44.192302 28211 raft_consensus.cc:740] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:44.168952 26954 catalog_manager.cc:5697] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:44.194989 26954 catalog_manager.cc:5697] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:44.196548 28402 raft_consensus.cc:3055] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:44.197038 28402 raft_consensus.cc:740] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:44.197341 26954 catalog_manager.cc:5697] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:44.199185 26954 catalog_manager.cc:5697] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:44.200482 28211 consensus_queue.cc:260] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:44.201006 26954 catalog_manager.cc:5697] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:44.201915 28402 consensus_queue.cc:260] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:44.204048 28402 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:44.204389 28304 raft_consensus.cc:2749] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:46:44.204622 28211 raft_consensus.cc:3060] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:44.205255 28304 consensus_queue.cc:1048] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:44.203919 27063 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:44.211897 27063 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:44.218287 28409 raft_consensus.cc:2749] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:44.219259 27063 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:44.220944 27063 leader_election.cc:304] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:44.222345 28409 raft_consensus.cc:2749] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:46:44.223248 28409 raft_consensus.cc:2749] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:44.214445 28304 raft_consensus.cc:2749] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:44.229136 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:44.230908 27213 raft_consensus.cc:1275] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260807 01:46:44.232384 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:44.233268 27337 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:44.234860 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:44.235903 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:44.237632 27170 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:44.238692 27063 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:44.239323 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:44.241752 28235 raft_consensus.cc:2749] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:46:44.250856 28409 raft_consensus.cc:2749] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:46:44.272013 28445 consensus_queue.cc:1059] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:44.274396 28233 raft_consensus.cc:3055] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:44.274834 28233 raft_consensus.cc:740] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:44.275781 28233 consensus_queue.cc:260] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:44.278766 28233 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:44.282608 27170 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:44.286564 27227 raft_consensus.cc:1275] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:44.288818 28238 raft_consensus.cc:2749] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:46:44.289503 27170 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:44.289824 28335 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:44.293358 28335 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:44.295910 28335 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:44.298491 28335 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:44.300093 28235 consensus_queue.cc:1059] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:44.299774 28335 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:44.301498 28456 raft_consensus.cc:3055] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:44.301891 28456 raft_consensus.cc:740] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:44.302860 28456 consensus_queue.cc:260] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:44.303316 28335 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:44.298323 28238 raft_consensus.cc:2804] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:46:44.306110 28238 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:44.307081 28238 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:44.304208 28456 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:44.314313 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:44.315272 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:44.317396 27102 raft_consensus.cc:1275] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260807 01:46:44.318451 28144 consensus_queue.cc:1059] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:46:44.319644 28144 raft_consensus.cc:3055] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:46:44.320003 27100 raft_consensus.cc:1275] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260807 01:46:44.317456 27216 raft_consensus.cc:1275] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260807 01:46:44.320016 28144 raft_consensus.cc:740] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:44.323544 27287 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:44.324360 28335 raft_consensus.cc:2749] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:46:44.324967 28144 consensus_queue.cc:260] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:44.325680 27287 leader_election.cc:304] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:44.326606 28144 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:44.328140 28304 consensus_queue.cc:1048] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 01:46:44.328630 28335 raft_consensus.cc:2749] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:46:44.323974 27170 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:44.328934 28304 consensus_queue.cc:1243] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer 3478729505b046c2a0051c2d0130873f log is divergent from this leader: its last log entry 1.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20260807 01:46:44.330017 28445 raft_consensus.cc:2749] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:44.330240 27287 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:44.331142 28335 raft_consensus.cc:2749] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:44.332541 27287 leader_election.cc:400] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 3. Current term is 4.
I20260807 01:46:44.331324 27170 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:44.333801 28445 raft_consensus.cc:2804] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:46:44.334257 28445 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:46:44.334704 28445 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:44.323441 26949 catalog_manager.cc:5697] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 3, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:44.340781 28505 consensus_queue.cc:1048] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:44.341003 28335 consensus_queue.cc:1048] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:44.343078 27287 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:44.345492 27287 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:44.353945 28394 raft_consensus.cc:2749] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:46:44.354784 26949 catalog_manager.cc:5697] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
W20260807 01:46:44.356417 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:46:44.356940 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:46:44.357625 27213 pending_rounds.cc:85] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f: Aborting all ops after (but not including) 0
I20260807 01:46:44.344420 28335 raft_consensus.cc:2749] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:46:44.357993 27213 pending_rounds.cc:107] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260807 01:46:44.360270 27213 raft_consensus.cc:2889] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 01:46:44.375342 27105 raft_consensus.cc:1275] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260807 01:46:44.380486 28238 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:44.388067 28445 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:44.389389 27216 raft_consensus.cc:1240] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 3. Current term is 4. Ops: []
I20260807 01:46:44.400434 28402 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:44.401252 28402 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:44.402559 27287 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:44.402704 28233 consensus_queue.cc:1048] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:44.403600 28335 raft_consensus.cc:2749] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:46:44.404148 28402 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:44.404732 28409 consensus_queue.cc:1059] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:46:44.409978 28409 raft_consensus.cc:3055] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:46:44.410432 28409 raft_consensus.cc:740] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:44.411300 28409 consensus_queue.cc:260] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:44.412556 28409 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:44.426136 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:44.426447 27330 raft_consensus.cc:1240] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 1. Current term is 2. Ops: []
I20260807 01:46:44.417454 28335 consensus_queue.cc:1048] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260807 01:46:44.434763 27325 raft_consensus.cc:1240] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3. Ops: []
I20260807 01:46:44.445482 28445 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:44.449087 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:44.450922 28238 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:44.452541 27216 raft_consensus.cc:1240] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 3. Current term is 4. Ops: []
I20260807 01:46:44.454684 28505 consensus_queue.cc:1059] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:46:44.455680 28304 raft_consensus.cc:3055] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:46:44.456059 28304 raft_consensus.cc:740] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:44.456950 28304 consensus_queue.cc:260] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:44.458758 28304 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:44.460611 27101 raft_consensus.cc:2468] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:46:44.467799 27172 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:44.469182 27328 raft_consensus.cc:1275] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260807 01:46:44.473498 26954 catalog_manager.cc:5697] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 0 to 2, leader changed from <none> to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 2 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:44.477744 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:44.478189 26954 catalog_manager.cc:5697] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 0 to 2, leader changed from <none> to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 2 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:44.479084 27328 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:44.479867 28238 raft_consensus.cc:2804] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:46:44.482291 28238 raft_consensus.cc:697] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:44.483253 28238 consensus_queue.cc:237] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:44.491470 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:44.501101 27063 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:44.501581 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:44.492734 27325 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:44.505064 27063 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:44.505908 28489 raft_consensus.cc:2804] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:44.506412 28489 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:44.506691 28501 raft_consensus.cc:2749] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:44.512403 27063 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:44.515684 28304 raft_consensus.cc:2749] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:44.518808 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:44.519366 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:44.522621 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:44.506805 28489 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:44.534230 28333 consensus_queue.cc:1059] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:44.534771 28446 consensus_queue.cc:1059] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:46:44.547545 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:44.549367 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:44.535917 28233 raft_consensus.cc:3055] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:46:44.564039 27287 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:44.568678 28402 raft_consensus.cc:2749] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:44.569031 28335 consensus_queue.cc:1048] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 01:46:44.570509 28394 consensus_queue.cc:1048] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 01:46:44.572135 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:46:44.574954 28456 raft_consensus.cc:3055] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:46:44.575374 28456 raft_consensus.cc:740] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:44.576262 28456 consensus_queue.cc:260] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:44.577622 28456 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
W20260807 01:46:44.578765 27067 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:46:44.562914 28233 raft_consensus.cc:740] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:44.560135 26950 catalog_manager.cc:5697] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 0 to 2, leader changed from <none> to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:44.588610 28233 consensus_queue.cc:260] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:44.595963 28233 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
W20260807 01:46:44.615741 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 61381.4us
I20260807 01:46:44.638015 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:44.623137 28235 consensus_queue.cc:1048] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260807 01:46:44.644799 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:44.651994 27337 raft_consensus.cc:1275] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260807 01:46:44.658130 27287 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:44.660328 27287 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:44.663015 27287 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:46:44.667127 27063 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:44.650099 27325 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:46:44.662412 28402 raft_consensus.cc:2749] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:44.666340 28335 raft_consensus.cc:2749] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:44.673056 28351 raft_consensus.cc:2749] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:46:44.682618 28394 raft_consensus.cc:2749] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:46:44.682963 28489 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:44.685278 27326 raft_consensus.cc:1240] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260807 01:46:44.685343 28489 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:44.680158 28409 consensus_queue.cc:1048] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.018s
I20260807 01:46:44.678879 27328 raft_consensus.cc:2468] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:46:44.700096 27338 raft_consensus.cc:1240] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260807 01:46:44.703418 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:44.679431 27063 leader_election.cc:304] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:44.707355 28409 raft_consensus.cc:2749] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:44.712498 27337 raft_consensus.cc:1275] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260807 01:46:44.735777 28536 log.cc:927] Time spent T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.056s user 0.000s sys 0.000s
I20260807 01:46:44.740923 27170 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
W20260807 01:46:44.742936 28535 log.cc:927] Time spent T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.066s user 0.000s sys 0.000s
I20260807 01:46:44.754297 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:44.754455 27063 leader_election.cc:304] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:44.760947 26950 catalog_manager.cc:5697] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 3 to 4, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 4 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:44.759500 28456 raft_consensus.cc:2804] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:46:44.766597 28456 raft_consensus.cc:697] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:44.764289 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:44.767663 28456 consensus_queue.cc:237] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:44.776321 28409 raft_consensus.cc:2749] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:46:44.777683 28304 consensus_queue.cc:1048] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.029s
I20260807 01:46:44.819612 27063 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:44.825562 27100 raft_consensus.cc:1240] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3. Ops: []
I20260807 01:46:44.835721 28409 raft_consensus.cc:2749] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:46:44.871985 28233 consensus_queue.cc:1059] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:46:44.886500 26954 catalog_manager.cc:5697] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 3 to 4, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 4 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:44.898900 28446 raft_consensus.cc:3055] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:46:44.918094 27102 raft_consensus.cc:1275] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:46:44.899365 28446 raft_consensus.cc:740] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:44.929272 28446 consensus_queue.cc:260] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:44.930753 28446 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:44.942356 28531 consensus_queue.cc:1048] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260807 01:46:44.978474 27330 raft_consensus.cc:1275] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:46:44.995286 28233 consensus_queue.cc:1048] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:45.065400 28235 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:45.065994 28235 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.068256 28235 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:45.073136 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:45.074025 27337 raft_consensus.cc:2468] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:46:45.075340 27170 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:45.076233 28233 raft_consensus.cc:2804] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:45.076682 28233 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:45.078969 28233 raft_consensus.cc:3060] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:45.092819 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:45.093896 27102 raft_consensus.cc:2393] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 2: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:45.105654 28233 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.119335 28233 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:45.121438 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:45.125972 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:45.126906 27105 raft_consensus.cc:2393] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 2: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:45.129511 27337 raft_consensus.cc:2468] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:46:45.130812 27170 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:45.132046 28233 raft_consensus.cc:2804] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:46:45.134402 28233 raft_consensus.cc:697] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:45.135568 28233 consensus_queue.cc:237] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.170571 26949 catalog_manager.cc:5697] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 1 to 2, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:45.279368 28552 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:45.279928 28552 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.282019 28552 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:45.284072 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:45.285714 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:45.286450 27212 raft_consensus.cc:2468] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:46:45.287985 27287 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:45.288861 28402 raft_consensus.cc:2804] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:45.289232 28402 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:45.289559 28402 raft_consensus.cc:3060] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:45.305994 28402 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.318219 28235 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:45.318889 28235 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.321847 28402 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:45.322891 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:45.328454 27105 raft_consensus.cc:2393] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 2: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:45.324555 28235 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:45.331387 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:45.331305 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:45.337181 27212 raft_consensus.cc:3060] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:45.341306 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:45.342051 27337 raft_consensus.cc:2468] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:46:45.343729 27337 raft_consensus.cc:1275] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:46:45.349638 27170 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:45.351584 28235 raft_consensus.cc:2804] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:45.355222 28235 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:45.355759 28235 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:45.360788 28402 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:45.361385 28402 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.362991 28233 consensus_queue.cc:1048] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:45.363371 28402 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:45.365759 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:45.366487 27105 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:46:45.368033 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:45.371340 27213 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:46:45.378044 27287 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:45.399420 27102 raft_consensus.cc:1275] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:46:45.400506 28402 raft_consensus.cc:2804] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:45.401000 28402 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:45.401444 28402 raft_consensus.cc:3060] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:45.401911 28333 consensus_queue.cc:1048] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:46:45.404186 28235 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.407861 27212 raft_consensus.cc:2468] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:46:45.409029 28235 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:45.419842 28402 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.424153 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:45.425477 27105 raft_consensus.cc:2393] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 2: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:45.424407 28402 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:45.464198 28233 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:45.464731 28233 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.466666 28233 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:45.476374 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:45.482487 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:45.482723 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:45.483402 27102 raft_consensus.cc:3060] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 3
I20260807 01:46:45.483196 27212 raft_consensus.cc:3060] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:45.491870 27102 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:46:45.500499 27287 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:45.501569 28402 raft_consensus.cc:2804] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:45.502296 27212 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:46:45.503010 28402 raft_consensus.cc:697] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:45.504612 28402 consensus_queue.cc:237] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.508654 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:45.516369 27337 raft_consensus.cc:2468] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:46:45.521679 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:45.522414 27333 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:46:45.546823 27287 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:45.547699 27170 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:45.549022 28235 raft_consensus.cc:2804] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:46:45.549134 27170 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:45.549546 28235 raft_consensus.cc:697] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:45.550868 28446 raft_consensus.cc:2804] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:45.551285 28446 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:45.550354 26954 catalog_manager.cc:5697] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:45.551643 28446 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:45.553097 28552 raft_consensus.cc:2804] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:45.553663 28552 raft_consensus.cc:697] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:45.554574 28552 consensus_queue.cc:237] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.550457 28235 consensus_queue.cc:237] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.565987 28446 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.574034 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:45.576685 28446 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:45.593525 26949 catalog_manager.cc:5697] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 1 to 2, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:45.594501 26954 catalog_manager.cc:5697] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:45.595954 28531 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:45.596154 28235 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:45.596540 28531 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.597679 28235 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.599850 28235 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:45.602741 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:45.603418 27337 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:45.609117 28409 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:45.609728 28409 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.611264 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:45.611786 28409 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:45.611901 27338 raft_consensus.cc:2468] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:46:45.614337 28531 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:45.615185 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:45.615939 27212 raft_consensus.cc:2468] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:45.617693 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:45.618604 27105 raft_consensus.cc:2393] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 2: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:45.620846 28565 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:45.621438 28565 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.623297 27064 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:45.625351 28491 raft_consensus.cc:2804] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:45.625808 28491 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:45.626189 28491 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:45.627414 27170 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:45.628229 28333 raft_consensus.cc:2804] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:45.628584 28333 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:45.628912 28333 raft_consensus.cc:3060] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:45.631554 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:45.647192 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:45.648139 27102 raft_consensus.cc:2468] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:46:45.651906 28333 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.653568 28491 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.669376 27105 raft_consensus.cc:2468] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:46:45.656118 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:45.658852 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:45.670742 27338 raft_consensus.cc:2468] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:46:45.673390 28491 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:45.675521 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:45.623667 28565 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:45.690721 27172 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:45.689162 27337 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:46:45.692802 28491 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:45.694380 28333 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:45.695303 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:45.696895 27105 raft_consensus.cc:2393] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 2: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:45.696973 27333 raft_consensus.cc:1275] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:46:45.698581 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:45.699380 27326 raft_consensus.cc:2393] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 2: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:45.697777 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:45.700279 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:45.701010 27338 raft_consensus.cc:2393] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 2: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:45.703510 27170 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:45.705401 28531 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:45.705940 28531 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.707404 27287 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:45.708789 28235 raft_consensus.cc:2804] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:45.709067 28552 raft_consensus.cc:2804] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:45.709342 28235 raft_consensus.cc:697] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:45.709510 28552 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:45.709937 28552 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:45.710484 28531 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:45.713162 27213 raft_consensus.cc:2468] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:45.714380 28235 consensus_queue.cc:237] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.715090 28446 consensus_queue.cc:1048] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:45.715786 27064 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:45.717558 28489 raft_consensus.cc:2804] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:46:45.717922 28552 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.720069 28552 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:45.718066 28489 raft_consensus.cc:697] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:45.725679 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:45.726631 27100 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:45.726512 28489 consensus_queue.cc:237] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.733520 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:45.736658 27102 raft_consensus.cc:1275] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:46:45.737864 28233 raft_consensus.cc:2804] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:45.738907 28233 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:45.739274 28233 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:45.740995 28333 consensus_queue.cc:1048] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:45.743134 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:45.743762 27337 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:46:45.752566 26954 catalog_manager.cc:5697] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 2, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:45.759977 26949 catalog_manager.cc:5697] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:45.778685 27330 raft_consensus.cc:2468] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:46:45.778968 27170 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:45.693451 28491 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.779812 28333 raft_consensus.cc:2804] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:46:45.780225 28333 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:45.780582 28333 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:45.782097 28491 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:45.735452 27100 raft_consensus.cc:2468] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:46:45.735991 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:45.783073 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:45.784077 27105 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:46:45.804404 28233 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.790557 28333 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.807325 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:45.811412 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:45.812026 27102 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:45.813349 28531 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:45.814136 28574 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:46:45.813956 28531 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.814697 28574 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.816171 28531 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:45.816830 28574 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:45.817059 28333 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:45.813982 27287 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:45.824990 28552 raft_consensus.cc:2804] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:45.825495 28552 raft_consensus.cc:697] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:45.826529 28552 consensus_queue.cc:237] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.831646 27170 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:45.832516 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:45.825373 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:45.833855 27212 raft_consensus.cc:2368] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 2. Current term is 3.
I20260807 01:46:45.836308 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:45.836949 27333 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 4
I20260807 01:46:45.836958 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:45.838089 27100 raft_consensus.cc:2368] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3.
I20260807 01:46:45.838696 27102 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:46:45.840866 28233 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:45.841774 28574 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:45.842360 28574 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.844872 28574 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:45.846572 27172 leader_election.cc:400] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3.
I20260807 01:46:45.846992 27172 leader_election.cc:403] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260807 01:46:45.849282 28531 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 3
I20260807 01:46:45.850427 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:45.851063 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:45.851529 28446 raft_consensus.cc:2804] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:46:45.851831 27336 raft_consensus.cc:2393] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 3: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:45.852111 28446 raft_consensus.cc:697] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:45.853157 28446 consensus_queue.cc:237] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.853808 28574 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:45.854346 28574 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.856628 28574 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:45.859972 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:45.860746 27337 raft_consensus.cc:2368] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 2. Current term is 3.
I20260807 01:46:45.867923 27063 leader_election.cc:400] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Vote denied by peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429) with higher term. Message: Invalid argument: T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 2. Current term is 3.
I20260807 01:46:45.868278 27063 leader_election.cc:403] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260807 01:46:45.868814 28491 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 3
I20260807 01:46:45.869004 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:45.869836 27338 raft_consensus.cc:2468] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:45.870888 27063 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:45.871666 28574 raft_consensus.cc:2804] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:45.872231 28574 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:45.871618 27172 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:45.872594 28574 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:45.873103 28579 raft_consensus.cc:2804] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:46:45.873790 28579 raft_consensus.cc:697] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:45.874701 28579 consensus_queue.cc:237] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.877246 28491 raft_consensus.cc:2749] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429) with higher term. Message: Invalid argument: T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 2. Current term is 3.
I20260807 01:46:45.872573 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:45.879060 27337 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:45.883011 28235 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:45.883726 28235 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.886981 27064 leader_election.cc:400] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 2. Current term is 3.
I20260807 01:46:45.887360 27064 leader_election.cc:403] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260807 01:46:45.892045 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:45.892832 27212 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:46:45.893936 28491 raft_consensus.cc:3060] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 3
I20260807 01:46:45.920274 28574 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.921064 26954 catalog_manager.cc:5697] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 1 to 2, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:45.927559 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:45.927840 27333 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:46:45.928771 28531 raft_consensus.cc:2749] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3.
I20260807 01:46:45.933240 27337 raft_consensus.cc:2468] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:46:45.932859 28574 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:45.938795 27170 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:45.938992 26954 catalog_manager.cc:5697] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:45.941087 28333 raft_consensus.cc:2804] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:45.932777 27064 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:46:45.941627 28333 raft_consensus.cc:697] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:45.943946 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:45.944314 28333 consensus_queue.cc:237] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.945825 28489 raft_consensus.cc:2804] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:45.946261 28489 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:45.946627 28489 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:45.949708 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:45.946718 28574 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:45.951380 28574 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.944876 27337 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:45.942831 28235 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:45.966343 28574 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:45.966638 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:45.967284 27338 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:46:45.971530 28491 raft_consensus.cc:2749] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 2. Current term is 3.
I20260807 01:46:45.971997 27170 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:45.966490 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:45.972591 27102 raft_consensus.cc:1275] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:45.973152 28235 raft_consensus.cc:2804] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:46:45.973786 28235 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:45.974324 28489 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.973089 28552 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:45.975834 27100 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:46:45.975922 28552 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:45.978446 28552 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:45.974265 28235 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:45.984444 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:45.985532 27212 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:45.990000 28489 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:45.994135 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:45.995078 27338 raft_consensus.cc:2393] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 2: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:45.999958 27064 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:46:46.000831 28409 raft_consensus.cc:2804] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:46:46.001319 28409 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:46.001730 28409 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:46:46.003551 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:46:46.003839 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:46.004487 27100 raft_consensus.cc:2393] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 3: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:46.005297 27216 raft_consensus.cc:1275] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:46.004487 27212 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:46:46.007341 27337 raft_consensus.cc:2468] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:46:46.014376 28235 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:46.013989 26954 catalog_manager.cc:5697] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 2 to 3, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 3 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:46.015863 26954 catalog_manager.cc:5697] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:46.017058 28552 consensus_queue.cc:1048] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:46.000883 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:46.021692 27338 raft_consensus.cc:2393] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 3: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:46.028414 27063 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:46.029526 28587 raft_consensus.cc:2804] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:46.030722 28587 raft_consensus.cc:697] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:46.031497 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:46.032235 27100 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:46.031637 28587 consensus_queue.cc:237] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:46.030781 28409 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:46.040139 28235 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:46.040297 28409 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:46.043534 28552 consensus_queue.cc:1048] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:46.044626 27287 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:46.046363 28552 raft_consensus.cc:2804] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:46.046759 28552 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:46:46.047178 28552 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:46.052789 27105 raft_consensus.cc:1240] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3. Ops: []
I20260807 01:46:46.054683 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:46.055428 27333 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:46.055142 28552 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:46.057337 28552 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:46.062260 27336 raft_consensus.cc:1240] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3. Ops: []
I20260807 01:46:46.062373 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:46.064488 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:46.065650 27105 raft_consensus.cc:2393] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 3: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:46:46.069252 28235 consensus_queue.cc:1059] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:46.070123 28235 raft_consensus.cc:3055] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:46:46.070530 28235 raft_consensus.cc:740] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:46.070963 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:46.071736 27337 raft_consensus.cc:2393] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 2: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:46:46.071470 28235 consensus_queue.cc:260] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:46.074831 28235 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:46.072526 26954 catalog_manager.cc:5697] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 2 to 3, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:46.084928 27333 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:46:46.086246 27170 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:46.087149 28580 raft_consensus.cc:2804] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:46:46.087646 28580 raft_consensus.cc:697] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:46.088469 28580 consensus_queue.cc:237] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:46.090869 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:46.097882 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:46.104974 27109 raft_consensus.cc:1275] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:46.106725 28552 consensus_queue.cc:1048] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:46.108453 27213 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:46.115492 27064 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:46:46.119904 27227 raft_consensus.cc:1240] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 2. Current term is 3. Ops: []
I20260807 01:46:46.139654 28409 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:46.140209 28409 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:46.108758 27100 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:46:46.130131 27216 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:46:46.143527 28409 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:46.144112 28489 raft_consensus.cc:2749] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:46:46.144901 27287 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:46.144922 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:46.145769 28565 raft_consensus.cc:2804] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:46.146253 27333 raft_consensus.cc:2468] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:46:46.146371 28565 raft_consensus.cc:697] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:46.147760 28565 consensus_queue.cc:237] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:46.152745 27063 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:46.156253 28489 raft_consensus.cc:2804] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:46.156960 28489 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:46.157332 28489 raft_consensus.cc:3060] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:46.164252 26949 catalog_manager.cc:5697] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:46.169787 27064 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:46:46.171746 28587 raft_consensus.cc:2804] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:46:46.173192 28587 raft_consensus.cc:697] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:46.174314 28587 consensus_queue.cc:237] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:46.179032 28409 consensus_queue.cc:1059] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:46.179880 28409 consensus_queue.cc:1243] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer 3478729505b046c2a0051c2d0130873f log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260807 01:46:46.180430 28491 raft_consensus.cc:3055] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:46:46.180804 28491 raft_consensus.cc:740] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:46.181615 28491 consensus_queue.cc:260] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:46.183439 28491 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:46.191157 27216 raft_consensus.cc:1275] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:46.197064 28552 consensus_queue.cc:1048] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:46.200368 27333 raft_consensus.cc:1275] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:46.202316 28580 consensus_queue.cc:1048] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:46.222462 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:46.233510 27101 raft_consensus.cc:1275] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:46.238049 28409 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:46.238644 28409 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:46.240986 28409 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:46.242900 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:46:46.243556 27337 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:46:46.241878 26949 catalog_manager.cc:5697] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 2, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:46.245924 27063 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:46.252218 28574 raft_consensus.cc:2804] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:46:46.252650 28574 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:46.252957 28574 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:46.272146 24592 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260807 01:46:46.275777 28446 consensus_queue.cc:1048] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:46.276983 28574 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:46.262460 26954 catalog_manager.cc:5697] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 2 to 3, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 3 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:46.281847 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:46.288542 27227 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:46:46.269300 28489 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:46.304188 28489 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:46.306859 28574 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:46.331225 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:46.331935 27338 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:46:46.345140 27338 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:46:46.350273 27099 raft_consensus.cc:1275] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:46.352849 27337 raft_consensus.cc:1275] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:46:46.345958 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:46:46.359617 28565 consensus_queue.cc:1048] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:46.360873 27333 raft_consensus.cc:3060] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:46.361649 28580 consensus_queue.cc:1048] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260807 01:46:46.364488 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 63309.7us
I20260807 01:46:46.367302 27063 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:46.368383 28608 raft_consensus.cc:2804] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:46:46.368875 28608 raft_consensus.cc:697] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:46.369819 28608 consensus_queue.cc:237] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:46.370244 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:46.371613 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:46.372161 27227 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
W20260807 01:46:46.378223 28610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:46:46.378791 27109 raft_consensus.cc:1275] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:46:46.385229 28235 consensus_queue.cc:1048] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:46:46.390355 27105 raft_consensus.cc:1240] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3. Ops: []
I20260807 01:46:46.390398 27227 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:46:46.403092 28565 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:46:46.403739 28565 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:46.405944 28565 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:46.413501 28235 consensus_queue.cc:1059] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:46.414439 28235 raft_consensus.cc:3055] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:46:46.414736 28235 raft_consensus.cc:740] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:46:46.414886 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:46:46.415566 27216 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:46:46.415505 28235 consensus_queue.cc:260] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:46.416651 28235 raft_consensus.cc:3060] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:46.419806 26949 catalog_manager.cc:5697] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 4, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 4 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:46.424043 27287 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:46:46.427834 28602 raft_consensus.cc:2804] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:46:46.428277 28602 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:46:46.428622 28602 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:46.433525 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:46:46.437292 28615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:46:46.442411 28602 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:46.445161 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:46:46.449671 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:46:46.450367 27212 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:46.460660 27227 raft_consensus.cc:1275] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:46.462471 28599 consensus_queue.cc:1048] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:46.491400 27212 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:46:46.449697 28602 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:46.495797 27333 raft_consensus.cc:2468] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:46:46.536980 27063 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:46:46.538228 28491 raft_consensus.cc:2804] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:46.538727 28491 raft_consensus.cc:697] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:46.539616 28491 consensus_queue.cc:237] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:46.546425 27287 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:46:46.549908 28602 raft_consensus.cc:2804] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:46:46.550495 28602 raft_consensus.cc:697] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:46:46.551414 28602 consensus_queue.cc:237] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:46.557947 27099 raft_consensus.cc:1275] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260807 01:46:46.560252 27099 pending_rounds.cc:85] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3: Aborting all ops after (but not including) 1
I20260807 01:46:46.558387 27326 raft_consensus.cc:1275] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:46.564002 27099 pending_rounds.cc:107] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260807 01:46:46.564462 27099 raft_consensus.cc:2889] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 01:46:46.567222 28446 consensus_queue.cc:1048] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:46.576921 27216 raft_consensus.cc:1275] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260807 01:46:46.577359 27216 pending_rounds.cc:85] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f: Aborting all ops after (but not including) 1
I20260807 01:46:46.577920 27216 pending_rounds.cc:107] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260807 01:46:46.578516 27216 raft_consensus.cc:2889] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 01:46:46.597404 28608 consensus_queue.cc:1048] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260807 01:46:46.602857 28501 consensus_queue.cc:1048] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:46.616667 27216 raft_consensus.cc:1240] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 2. Current term is 3. Ops: []
W20260807 01:46:46.600752 28628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 01:46:46.625114 28608 consensus_queue.cc:1059] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260807 01:46:46.627755 27326 raft_consensus.cc:1275] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:46.627849 28608 raft_consensus.cc:3055] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:46:46.629026 28608 raft_consensus.cc:740] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:46:46.629864 28608 consensus_queue.cc:260] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:46:46.631035 28608 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:46:46.633226 28579 consensus_queue.cc:1048] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:46.642295 26950 catalog_manager.cc:5697] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 2 to 3, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:46.653730 27216 raft_consensus.cc:1275] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:46:46.656180 27105 raft_consensus.cc:1275] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:46:46.656287 26949 catalog_manager.cc:5697] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 2 to 3, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 3 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:46:46.659672 28552 consensus_queue.cc:1048] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260807 01:46:46.667500 27333 raft_consensus.cc:1275] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:46:46.670277 28602 consensus_queue.cc:1048] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260807 01:46:46.671893 28531 consensus_queue.cc:1048] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260807 01:46:46.677440 24592 server_base.cc:1061] running on GCE node
I20260807 01:46:46.689939 24592 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 01:46:46.690349 24592 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 01:46:46.690618 24592 hybrid_clock.cc:648] HybridClock initialized: now 1786067206690599 us; error 0 us; skew 500 ppm
I20260807 01:46:46.691355 24592 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 01:46:46.726334 27099 raft_consensus.cc:1275] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:46:46.728924 28235 consensus_queue.cc:1048] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:46:46.844789 27337 raft_consensus.cc:1275] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260807 01:46:46.857683 28608 consensus_queue.cc:1048] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:46.867442 27227 raft_consensus.cc:1275] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260807 01:46:46.872702 28608 consensus_queue.cc:1048] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:46.878708 24592 webserver.cc:533] Webserver started at http://127.24.4.4:38843/ using document root <none> and password file <none>
I20260807 01:46:46.880291 24592 fs_manager.cc:362] Metadata directory not provided
I20260807 01:46:46.880980 24592 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 01:46:46.881696 24592 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 01:46:46.890220 24592 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-3-root/instance:
uuid: "57bb4985157447f0a3ab680fa0026f12"
format_stamp: "Formatted at 2026-08-07 01:46:46 on dist-test-slave-zj8q"
I20260807 01:46:46.953353 27338 raft_consensus.cc:1275] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:46.959754 28608 consensus_queue.cc:1048] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:46.981882 24592 fs_manager.cc:696] Time spent creating directory manager: real 0.090s user 0.008s sys 0.018s
I20260807 01:46:47.002780 28653 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 01:46:47.005584 24592 fs_manager.cc:730] Time spent opening block manager: real 0.020s user 0.003s sys 0.004s
I20260807 01:46:47.006027 24592 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-3-root
uuid: "57bb4985157447f0a3ab680fa0026f12"
format_stamp: "Formatted at 2026-08-07 01:46:46 on dist-test-slave-zj8q"
I20260807 01:46:47.006433 24592 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 01:46:47.011901 27227 raft_consensus.cc:1275] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260807 01:46:47.012353 27227 pending_rounds.cc:85] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f: Aborting all ops after (but not including) 1
I20260807 01:46:47.012576 27227 pending_rounds.cc:107] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260807 01:46:47.012813 27227 raft_consensus.cc:2889] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 01:46:47.014839 28491 consensus_queue.cc:1048] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:47.066363 27212 raft_consensus.cc:1275] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:46:47.071612 28552 consensus_queue.cc:1048] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:47.109338 27109 raft_consensus.cc:1275] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260807 01:46:47.109809 27109 pending_rounds.cc:85] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3: Aborting all ops after (but not including) 1
I20260807 01:46:47.110064 27109 pending_rounds.cc:107] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260807 01:46:47.110641 27109 raft_consensus.cc:2889] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 01:46:47.114369 28552 consensus_queue.cc:1048] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:47.186488 24592 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 01:46:47.188769 24592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 01:46:47.201412 24592 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 01:46:47.237262 24592 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 01:46:47.237582 24592 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 01:46:47.237890 24592 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 01:46:47.238174 24592 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 01:46:47.983484 24592 rpc_server.cc:307] RPC server started. Bound to: 127.24.4.4:46225
I20260807 01:46:47.983831 28748 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.24.4.4:46225 every 8 connection(s)
I20260807 01:46:48.230887 24592 auto_leader_rebalancer-test.cc:581] add a tserver: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:46:48.248360 28750 heartbeater.cc:344] Connected to a master server at 127.24.4.62:35633
I20260807 01:46:48.248899 28750 heartbeater.cc:461] Registering TS with master...
I20260807 01:46:48.249969 28750 heartbeater.cc:507] Master 127.24.4.62:35633 requested a full tablet report, sending...
I20260807 01:46:48.252444 26949 ts_manager.cc:194] Registered new tserver with Master: 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:46:48.254480 26949 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60866
I20260807 01:46:48.835891 27216 consensus_queue.cc:237] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:48.868903 27105 raft_consensus.cc:1275] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:46:48.874498 28446 consensus_queue.cc:1048] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:46:48.875604 27333 raft_consensus.cc:1275] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:46:48.878389 28765 consensus_queue.cc:1048] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260807 01:46:49.146818 27175 consensus_peers.cc:597] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6420178dae2448f78216552b1b03c6e6. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:49.152586 28446 raft_consensus.cc:2955] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:49.195811 27099 raft_consensus.cc:2955] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:49.231976 26949 catalog_manager.cc:5697] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 3 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:49.239859 27333 raft_consensus.cc:2955] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:49.264882 28750 heartbeater.cc:499] Master 127.24.4.62:35633 was elected leader, sending a full tablet report...
I20260807 01:46:49.367090 27333 consensus_queue.cc:237] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:49.392969 27227 raft_consensus.cc:1275] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 01:46:49.394654 27099 raft_consensus.cc:1275] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 01:46:49.412925 28757 consensus_queue.cc:1048] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:49.423197 28783 consensus_queue.cc:1048] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:49.550571 28809 ts_tablet_manager.cc:933] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:49.577203 28788 raft_consensus.cc:2955] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:49.579945 27099 raft_consensus.cc:2955] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:49.620051 27216 raft_consensus.cc:2955] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:49.624106 28809 tablet_copy_client.cc:323] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.2:34855
I20260807 01:46:49.630988 26954 catalog_manager.cc:5697] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
W20260807 01:46:49.641767 27289 consensus_peers.cc:597] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a0e280c976844349b4874a479aa08de2. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:49.692631 27336 consensus_queue.cc:237] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:49.735225 27213 raft_consensus.cc:1275] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:46:49.736970 28788 consensus_queue.cc:1048] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:46:49.737675 27099 raft_consensus.cc:1275] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260807 01:46:49.740365 27289 consensus_peers.cc:597] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d53e096d3ad04a5bb333660000313179. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:49.746506 28788 consensus_queue.cc:1048] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:46:49.775229 28757 raft_consensus.cc:2955] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:49.780570 27250 tablet_copy_service.cc:140] P 3478729505b046c2a0051c2d0130873f: Received BeginTabletCopySession request for tablet 6420178dae2448f78216552b1b03c6e6 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:41230)
I20260807 01:46:49.781311 27250 tablet_copy_service.cc:161] P 3478729505b046c2a0051c2d0130873f: Beginning new tablet copy session on tablet 6420178dae2448f78216552b1b03c6e6 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:41230: session id = 57bb4985157447f0a3ab680fa0026f12-6420178dae2448f78216552b1b03c6e6
I20260807 01:46:49.784217 27216 raft_consensus.cc:2955] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:49.796033 27250 tablet_copy_source_session.cc:215] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:46:49.796715 27101 raft_consensus.cc:2955] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:49.805476 28809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6420178dae2448f78216552b1b03c6e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:49.819128 26950 catalog_manager.cc:5697] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 3 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:49.822032 27330 consensus_queue.cc:237] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:49.865942 27216 raft_consensus.cc:1275] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 01:46:49.866446 27101 raft_consensus.cc:1275] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 01:46:49.869315 28757 consensus_queue.cc:1048] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:49.871263 28757 consensus_queue.cc:1048] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260807 01:46:49.874749 27289 consensus_peers.cc:597] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bf9ccfe7f7e64481b739c6efef03b95a. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:49.885932 28809 tablet_copy_client.cc:806] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:46:49.893447 28809 tablet_copy_client.cc:670] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:46:49.897480 28788 raft_consensus.cc:2955] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:49.901918 27216 raft_consensus.cc:2955] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:49.906886 28809 tablet_copy_client.cc:538] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:46:49.910640 27101 raft_consensus.cc:2955] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:49.932360 28809 tablet_bootstrap.cc:492] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:46:49.932574 26950 catalog_manager.cc:5697] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:50.011933 28826 ts_tablet_manager.cc:933] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:50.063158 28809 tablet_bootstrap.cc:492] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:46:50.090293 28809 tablet_bootstrap.cc:492] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:46:50.091794 28809 ts_tablet_manager.cc:1403] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.160s user 0.028s sys 0.028s
I20260807 01:46:50.116585 28826 tablet_copy_client.cc:323] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.3:42429
I20260807 01:46:50.111943 28809 raft_consensus.cc:359] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:50.139809 28809 raft_consensus.cc:740] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:46:50.147338 28809 consensus_queue.cc:260] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:50.189507 27351 tablet_copy_service.cc:140] P fcf4e45defb94276bcfa68392711dade: Received BeginTabletCopySession request for tablet a0e280c976844349b4874a479aa08de2 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:36604)
I20260807 01:46:50.190243 27351 tablet_copy_service.cc:161] P fcf4e45defb94276bcfa68392711dade: Beginning new tablet copy session on tablet a0e280c976844349b4874a479aa08de2 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:36604: session id = 57bb4985157447f0a3ab680fa0026f12-a0e280c976844349b4874a479aa08de2
I20260807 01:46:50.197446 28809 ts_tablet_manager.cc:1434] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.105s user 0.011s sys 0.017s
I20260807 01:46:50.201077 27250 tablet_copy_service.cc:342] P 3478729505b046c2a0051c2d0130873f: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-6420178dae2448f78216552b1b03c6e6 received from {username='slave'} at 127.0.0.1:41230
I20260807 01:46:50.201638 27250 tablet_copy_service.cc:434] P 3478729505b046c2a0051c2d0130873f: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-6420178dae2448f78216552b1b03c6e6 on tablet 6420178dae2448f78216552b1b03c6e6 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:46:50.237556 28809 ts_tablet_manager.cc:933] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:50.276052 27351 tablet_copy_source_session.cc:215] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:46:50.292093 28826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0e280c976844349b4874a479aa08de2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:50.323963 28809 tablet_copy_client.cc:323] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.3:42429
I20260807 01:46:50.329402 27351 tablet_copy_service.cc:140] P fcf4e45defb94276bcfa68392711dade: Received BeginTabletCopySession request for tablet d53e096d3ad04a5bb333660000313179 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:36604)
I20260807 01:46:50.329988 27351 tablet_copy_service.cc:161] P fcf4e45defb94276bcfa68392711dade: Beginning new tablet copy session on tablet d53e096d3ad04a5bb333660000313179 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:36604: session id = 57bb4985157447f0a3ab680fa0026f12-d53e096d3ad04a5bb333660000313179
I20260807 01:46:50.339982 27351 tablet_copy_source_session.cc:215] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:46:50.347313 28826 tablet_copy_client.cc:806] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:46:50.348891 28809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d53e096d3ad04a5bb333660000313179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:50.351161 28826 tablet_copy_client.cc:670] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:46:50.364863 28826 tablet_copy_client.cc:538] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:46:50.395651 28826 tablet_bootstrap.cc:492] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:46:50.416738 28809 tablet_copy_client.cc:806] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:46:50.419174 28809 tablet_copy_client.cc:670] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:46:50.428436 28809 tablet_copy_client.cc:538] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:46:50.435912 28836 ts_tablet_manager.cc:933] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:50.456964 28836 tablet_copy_client.cc:323] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.3:42429
I20260807 01:46:50.460067 27351 tablet_copy_service.cc:140] P fcf4e45defb94276bcfa68392711dade: Received BeginTabletCopySession request for tablet bf9ccfe7f7e64481b739c6efef03b95a from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:36604)
I20260807 01:46:50.460852 27351 tablet_copy_service.cc:161] P fcf4e45defb94276bcfa68392711dade: Beginning new tablet copy session on tablet bf9ccfe7f7e64481b739c6efef03b95a from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:36604: session id = 57bb4985157447f0a3ab680fa0026f12-bf9ccfe7f7e64481b739c6efef03b95a
I20260807 01:46:50.477277 28809 tablet_bootstrap.cc:492] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:46:50.523367 27351 tablet_copy_source_session.cc:215] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:46:50.523423 28826 tablet_bootstrap.cc:492] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:46:50.529695 28826 tablet_bootstrap.cc:492] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:46:50.530592 28836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf9ccfe7f7e64481b739c6efef03b95a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:50.530601 28826 ts_tablet_manager.cc:1403] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.135s user 0.026s sys 0.007s
I20260807 01:46:50.552932 28826 raft_consensus.cc:359] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:50.554906 28826 raft_consensus.cc:740] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:46:50.559722 28826 consensus_queue.cc:260] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:50.561199 28715 raft_consensus.cc:1217] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3] Dedup: 3.3->[]
I20260807 01:46:50.564862 28836 tablet_copy_client.cc:806] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:46:50.570932 28836 tablet_copy_client.cc:670] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:46:50.567386 28826 ts_tablet_manager.cc:1434] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.032s user 0.008s sys 0.001s
I20260807 01:46:50.575711 27351 tablet_copy_service.cc:342] P fcf4e45defb94276bcfa68392711dade: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-a0e280c976844349b4874a479aa08de2 received from {username='slave'} at 127.0.0.1:36604
I20260807 01:46:50.576403 27351 tablet_copy_service.cc:434] P fcf4e45defb94276bcfa68392711dade: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-a0e280c976844349b4874a479aa08de2 on tablet a0e280c976844349b4874a479aa08de2 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:46:50.608865 28836 tablet_copy_client.cc:538] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:46:50.641996 28836 tablet_bootstrap.cc:492] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:46:50.671082 28809 tablet_bootstrap.cc:492] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:46:50.673234 28809 tablet_bootstrap.cc:492] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:46:50.677174 28809 ts_tablet_manager.cc:1403] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.200s user 0.034s sys 0.005s
I20260807 01:46:50.689879 28809 raft_consensus.cc:359] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:50.693840 28809 raft_consensus.cc:740] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:46:50.694669 28809 consensus_queue.cc:260] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:50.708631 28809 ts_tablet_manager.cc:1434] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.028s user 0.007s sys 0.000s
I20260807 01:46:50.715126 27350 tablet_copy_service.cc:342] P fcf4e45defb94276bcfa68392711dade: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-d53e096d3ad04a5bb333660000313179 received from {username='slave'} at 127.0.0.1:36604
I20260807 01:46:50.715646 27350 tablet_copy_service.cc:434] P fcf4e45defb94276bcfa68392711dade: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-d53e096d3ad04a5bb333660000313179 on tablet d53e096d3ad04a5bb333660000313179 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:46:50.746245 28715 raft_consensus.cc:1217] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3] Dedup: 3.3->[]
I20260807 01:46:50.858207 28715 raft_consensus.cc:1217] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260807 01:46:50.918399 28836 tablet_bootstrap.cc:492] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:46:50.931375 28836 tablet_bootstrap.cc:492] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:46:50.935333 28836 ts_tablet_manager.cc:1403] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.294s user 0.023s sys 0.003s
I20260807 01:46:50.969871 28836 raft_consensus.cc:359] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:50.998970 28836 raft_consensus.cc:740] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:46:51.016966 28836 consensus_queue.cc:260] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:51.036865 28836 ts_tablet_manager.cc:1434] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.080s user 0.006s sys 0.001s
I20260807 01:46:51.052419 27350 tablet_copy_service.cc:342] P fcf4e45defb94276bcfa68392711dade: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-bf9ccfe7f7e64481b739c6efef03b95a received from {username='slave'} at 127.0.0.1:36604
I20260807 01:46:51.052917 27350 tablet_copy_service.cc:434] P fcf4e45defb94276bcfa68392711dade: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-bf9ccfe7f7e64481b739c6efef03b95a on tablet bf9ccfe7f7e64481b739c6efef03b95a with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:46:51.103384 28580 raft_consensus.cc:1064] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:46:51.107149 28580 consensus_queue.cc:237] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:46:51.121328 28715 raft_consensus.cc:1275] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 01:46:51.132572 28446 consensus_queue.cc:1048] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:46:51.147590 28843 raft_consensus.cc:1064] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:46:51.150785 28843 consensus_queue.cc:237] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:46:51.161692 27338 raft_consensus.cc:1275] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 01:46:51.166570 28794 consensus_queue.cc:1048] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:46:51.177562 27101 raft_consensus.cc:1275] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 01:46:51.183260 28794 consensus_queue.cc:1048] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:46:51.232918 28787 raft_consensus.cc:2955] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.310568 27101 raft_consensus.cc:2955] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.337970 28715 raft_consensus.cc:1275] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 01:46:51.340466 28813 consensus_queue.cc:1048] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:46:51.404733 28714 raft_consensus.cc:2955] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.409487 26950 catalog_manager.cc:5697] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.429828 28843 raft_consensus.cc:1064] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:46:51.432448 28843 consensus_queue.cc:237] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:46:51.445709 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 62266.9us
I20260807 01:46:51.485831 27109 raft_consensus.cc:1275] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 01:46:51.485946 27227 raft_consensus.cc:1275] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 01:46:51.495735 28715 raft_consensus.cc:1275] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:46:51.498314 28843 consensus_queue.cc:1048] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:46:51.522119 28857 consensus_queue.cc:1048] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260807 01:46:51.560698 27337 raft_consensus.cc:2955] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.563592 27227 raft_consensus.cc:1275] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:46:51.564563 27109 raft_consensus.cc:1275] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:46:51.576709 28715 raft_consensus.cc:1217] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260807 01:46:51.579272 28813 consensus_queue.cc:1048] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:46:51.582664 28813 consensus_queue.cc:1048] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:46:51.599400 28813 consensus_queue.cc:1048] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260807 01:46:51.626308 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 60207.3us
I20260807 01:46:51.654393 28813 raft_consensus.cc:2955] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.659276 27099 raft_consensus.cc:2955] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.669476 27216 raft_consensus.cc:2955] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.668556 28783 raft_consensus.cc:2955] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.673360 27109 raft_consensus.cc:2955] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.693017 27229 raft_consensus.cc:2955] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.730886 28715 raft_consensus.cc:2955] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.737685 28714 raft_consensus.cc:2955] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.778914 26950 catalog_manager.cc:5697] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.788247 26950 catalog_manager.cc:5697] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.796412 27216 consensus_queue.cc:237] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:46:51.856242 28715 raft_consensus.cc:1275] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 01:46:51.859393 28446 consensus_queue.cc:1048] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:46:51.865990 27338 consensus_queue.cc:237] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:46:51.870285 27109 raft_consensus.cc:1275] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 01:46:51.877611 28869 raft_consensus.cc:2955] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.880419 28715 raft_consensus.cc:2955] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.876897 27337 consensus_queue.cc:237] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:46:51.888562 28714 raft_consensus.cc:1275] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:46:51.890403 28783 consensus_queue.cc:1048] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:46:51.896864 28794 consensus_queue.cc:1048] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:46:51.901048 27109 raft_consensus.cc:1275] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:46:51.908186 28865 raft_consensus.cc:2955] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.909333 28714 raft_consensus.cc:2955] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.920077 28713 raft_consensus.cc:1275] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 01:46:51.920248 26950 catalog_manager.cc:5697] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 4 to 5, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New cstate: current_term: 3 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.922863 28843 consensus_queue.cc:1048] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:46:51.937518 28863 raft_consensus.cc:2955] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.929517 28813 consensus_queue.cc:1048] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:46:51.939561 28711 raft_consensus.cc:2955] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.945989 27317 tablet_service.cc:1558] Processing DeleteTablet for tablet 6420178dae2448f78216552b1b03c6e6 with delete_type TABLET_DATA_TOMBSTONED (TS fcf4e45defb94276bcfa68392711dade not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:59474
I20260807 01:46:51.955092 27099 raft_consensus.cc:2955] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.976118 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6420178dae2448f78216552b1b03c6e6 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:46:51.988903 28881 tablet_replica.cc:333] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade: stopping tablet replica
I20260807 01:46:51.992249 26950 catalog_manager.cc:5697] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 3 to 4, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:51.993291 28881 raft_consensus.cc:2243] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:51.994692 28881 raft_consensus.cc:2272] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:51.997723 27216 raft_consensus.cc:1275] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 01:46:52.021592 27101 raft_consensus.cc:2955] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:52.035913 26931 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a0e280c976844349b4874a479aa08de2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:46:52.040987 28788 raft_consensus.cc:1064] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:46:52.046761 28788 consensus_queue.cc:237] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:46:52.068009 27199 tablet_service.cc:1558] Processing DeleteTablet for tablet a0e280c976844349b4874a479aa08de2 with delete_type TABLET_DATA_TOMBSTONED (TS 3478729505b046c2a0051c2d0130873f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53780
I20260807 01:46:52.078711 26931 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d53e096d3ad04a5bb333660000313179 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:46:52.102870 28881 ts_tablet_manager.cc:1916] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:46:52.104094 26952 catalog_manager.cc:5697] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index 4 to 5, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New cstate: current_term: 3 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:46:52.110277 28882 tablet_replica.cc:333] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f: stopping tablet replica
I20260807 01:46:52.126919 27088 tablet_service.cc:1558] Processing DeleteTablet for tablet d53e096d3ad04a5bb333660000313179 with delete_type TABLET_DATA_TOMBSTONED (TS 3262a9d183914b699227fdba0ca5bed3 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:41078
I20260807 01:46:52.144418 28882 raft_consensus.cc:2243] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:52.152828 28711 raft_consensus.cc:1275] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:46:52.160318 28882 raft_consensus.cc:2272] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:52.222764 27216 raft_consensus.cc:1275] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:46:52.227241 27101 raft_consensus.cc:1275] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:46:52.234617 28882 ts_tablet_manager.cc:1916] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:46:52.251776 28788 consensus_queue.cc:1048] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260807 01:46:52.265630 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 43644us
I20260807 01:46:52.270118 28884 tablet_replica.cc:333] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3: stopping tablet replica
I20260807 01:46:52.265275 28863 consensus_queue.cc:1048] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260807 01:46:52.321220 28881 ts_tablet_manager.cc:1929] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260807 01:46:52.327901 28881 log.cc:1199] T 6420178dae2448f78216552b1b03c6e6 P fcf4e45defb94276bcfa68392711dade: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-2-root/wals/6420178dae2448f78216552b1b03c6e6
W20260807 01:46:52.335197 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 104059us
I20260807 01:46:52.336331 28788 consensus_queue.cc:1048] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260807 01:46:52.342583 26931 catalog_manager.cc:5028] TS fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): tablet 6420178dae2448f78216552b1b03c6e6 (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:46:52.380030 28843 consensus_queue.cc:1048] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:46:52.413648 28884 raft_consensus.cc:2243] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:52.415355 28884 raft_consensus.cc:2272] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:52.418143 27229 raft_consensus.cc:2955] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:52.429003 28884 ts_tablet_manager.cc:1916] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:46:52.477634 28882 ts_tablet_manager.cc:1929] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 01:46:52.486967 28882 log.cc:1199] T a0e280c976844349b4874a479aa08de2 P 3478729505b046c2a0051c2d0130873f: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-1-root/wals/a0e280c976844349b4874a479aa08de2
I20260807 01:46:52.491295 26932 catalog_manager.cc:5028] TS 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): tablet a0e280c976844349b4874a479aa08de2 (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:46:52.620687 28863 raft_consensus.cc:2955] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:52.647469 28715 raft_consensus.cc:2955] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:52.653805 26950 catalog_manager.cc:5697] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:46:52.720564 27101 raft_consensus.cc:2955] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:52.730554 27326 consensus_queue.cc:237] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:46:52.740625 28715 raft_consensus.cc:1275] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:46:52.742600 28863 consensus_queue.cc:1048] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:46:52.742588 27229 raft_consensus.cc:2955] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:52.766326 27213 raft_consensus.cc:1275] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:46:52.771202 28843 raft_consensus.cc:2955] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:52.775977 28715 raft_consensus.cc:2955] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:52.784960 28884 ts_tablet_manager.cc:1929] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260807 01:46:52.785795 28884 log.cc:1199] T d53e096d3ad04a5bb333660000313179 P 3262a9d183914b699227fdba0ca5bed3: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-0-root/wals/d53e096d3ad04a5bb333660000313179
I20260807 01:46:52.787878 26932 catalog_manager.cc:5028] TS 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): tablet d53e096d3ad04a5bb333660000313179 (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:46:52.811966 26931 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bf9ccfe7f7e64481b739c6efef03b95a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:46:52.824105 28788 consensus_queue.cc:1048] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:46:52.818326 26950 catalog_manager.cc:5697] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:52.845732 27088 tablet_service.cc:1558] Processing DeleteTablet for tablet bf9ccfe7f7e64481b739c6efef03b95a with delete_type TABLET_DATA_TOMBSTONED (TS 3262a9d183914b699227fdba0ca5bed3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41078
I20260807 01:46:52.848657 28884 tablet_replica.cc:333] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3: stopping tablet replica
I20260807 01:46:52.849737 28884 raft_consensus.cc:2243] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:52.853821 27229 raft_consensus.cc:2955] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:52.856634 28884 raft_consensus.cc:2272] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:52.882817 28884 ts_tablet_manager.cc:1916] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:46:52.950556 28884 ts_tablet_manager.cc:1929] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 01:46:52.951088 28884 log.cc:1199] T bf9ccfe7f7e64481b739c6efef03b95a P 3262a9d183914b699227fdba0ca5bed3: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-0-root/wals/bf9ccfe7f7e64481b739c6efef03b95a
I20260807 01:46:52.953045 26932 catalog_manager.cc:5028] TS 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): tablet bf9ccfe7f7e64481b739c6efef03b95a (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:46:53.976649 27105 consensus_queue.cc:237] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:53.984845 27330 raft_consensus.cc:1275] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 01:46:53.988221 27216 raft_consensus.cc:1275] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 01:46:53.993032 28899 consensus_queue.cc:1048] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260807 01:46:54.050817 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 57832.3us
I20260807 01:46:54.061879 28834 consensus_queue.cc:1048] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
W20260807 01:46:54.110976 27067 consensus_peers.cc:597] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 51cef821f0284d8e9369e98d38f80e73. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:54.112116 27229 raft_consensus.cc:2955] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:54.104809 28908 raft_consensus.cc:2955] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:54.115968 27338 raft_consensus.cc:2955] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
W20260807 01:46:54.208335 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 69291.9us
I20260807 01:46:54.216646 27227 consensus_queue.cc:237] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:54.225186 26952 catalog_manager.cc:5697] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 2 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260807 01:46:54.414372 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 131126us
W20260807 01:46:54.427134 27175 consensus_peers.cc:597] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8767bf1fc98c4effa8f3b5f5113afe40. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:54.451267 27327 raft_consensus.cc:1275] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:46:54.474602 27105 raft_consensus.cc:1275] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:46:54.483613 28446 consensus_queue.cc:1048] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260807 01:46:54.509428 28930 ts_tablet_manager.cc:933] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:46:54.540800 28898 raft_consensus.cc:2955] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:54.546959 28794 consensus_queue.cc:1048] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:46:54.548163 27101 raft_consensus.cc:2955] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:54.564731 28930 tablet_copy_client.cc:323] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.1:35499
I20260807 01:46:54.606626 26954 catalog_manager.cc:5697] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index -1 to 4, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 4 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:54.608551 27216 consensus_queue.cc:237] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
W20260807 01:46:54.648533 27175 consensus_peers.cc:597] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 94962ef95ca14973bc39a17ed76c0087. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:54.651618 27101 raft_consensus.cc:1275] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 01:46:54.655565 28446 consensus_queue.cc:1048] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260807 01:46:54.666296 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 43685.7us
I20260807 01:46:54.666451 27127 tablet_copy_service.cc:140] P 3262a9d183914b699227fdba0ca5bed3: Received BeginTabletCopySession request for tablet 51cef821f0284d8e9369e98d38f80e73 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:52556)
I20260807 01:46:54.667248 27127 tablet_copy_service.cc:161] P 3262a9d183914b699227fdba0ca5bed3: Beginning new tablet copy session on tablet 51cef821f0284d8e9369e98d38f80e73 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:52556: session id = 57bb4985157447f0a3ab680fa0026f12-51cef821f0284d8e9369e98d38f80e73
I20260807 01:46:54.672887 27326 raft_consensus.cc:1275] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 01:46:54.686175 27127 tablet_copy_source_session.cc:215] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:46:54.689380 28919 raft_consensus.cc:2955] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:54.691617 27101 raft_consensus.cc:2955] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:54.696461 28446 consensus_queue.cc:1048] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:54.712563 27333 raft_consensus.cc:2955] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:54.716269 28930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51cef821f0284d8e9369e98d38f80e73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:54.736935 28930 tablet_copy_client.cc:806] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:46:54.738047 28930 tablet_copy_client.cc:670] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:46:54.739831 27338 consensus_queue.cc:237] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:54.735533 26950 catalog_manager.cc:5697] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:54.744046 28930 tablet_copy_client.cc:538] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:46:54.744799 27330 raft_consensus.cc:2955] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:54.755973 28930 tablet_bootstrap.cc:492] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:46:54.794289 28930 tablet_bootstrap.cc:492] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:46:54.795408 28930 tablet_bootstrap.cc:492] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:46:54.796250 28930 ts_tablet_manager.cc:1403] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.041s user 0.016s sys 0.017s
W20260807 01:46:54.798653 27289 consensus_peers.cc:597] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2cfefe1ff8f14fb68678ac128e129d32. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:54.799767 28930 raft_consensus.cc:359] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:54.802450 28930 raft_consensus.cc:740] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:46:54.803452 27101 raft_consensus.cc:1275] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:46:54.803315 28930 consensus_queue.cc:260] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:54.803373 27229 raft_consensus.cc:1275] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:46:54.805138 28936 consensus_queue.cc:1048] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:54.807334 28936 consensus_queue.cc:1048] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:54.832111 28788 raft_consensus.cc:2955] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:54.844642 27101 raft_consensus.cc:2955] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:54.911262 28930 ts_tablet_manager.cc:1434] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.115s user 0.015s sys 0.039s
I20260807 01:46:54.912951 27216 raft_consensus.cc:2955] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:54.916457 27127 tablet_copy_service.cc:342] P 3262a9d183914b699227fdba0ca5bed3: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-51cef821f0284d8e9369e98d38f80e73 received from {username='slave'} at 127.0.0.1:52556
I20260807 01:46:54.916950 27127 tablet_copy_service.cc:434] P 3262a9d183914b699227fdba0ca5bed3: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-51cef821f0284d8e9369e98d38f80e73 on tablet 51cef821f0284d8e9369e98d38f80e73 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:46:54.934211 26950 catalog_manager.cc:5697] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 2 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:55.029646 28930 ts_tablet_manager.cc:933] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:55.048570 28930 tablet_copy_client.cc:323] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.2:34855
I20260807 01:46:55.051270 27250 tablet_copy_service.cc:140] P 3478729505b046c2a0051c2d0130873f: Received BeginTabletCopySession request for tablet 8767bf1fc98c4effa8f3b5f5113afe40 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:41230)
I20260807 01:46:55.051851 27250 tablet_copy_service.cc:161] P 3478729505b046c2a0051c2d0130873f: Beginning new tablet copy session on tablet 8767bf1fc98c4effa8f3b5f5113afe40 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:41230: session id = 57bb4985157447f0a3ab680fa0026f12-8767bf1fc98c4effa8f3b5f5113afe40
I20260807 01:46:55.062860 27250 tablet_copy_source_session.cc:215] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:46:55.068208 28930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8767bf1fc98c4effa8f3b5f5113afe40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:55.104979 28930 tablet_copy_client.cc:806] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:46:55.106237 28930 tablet_copy_client.cc:670] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:46:55.111212 28930 tablet_copy_client.cc:538] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:46:55.122253 28930 tablet_bootstrap.cc:492] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:46:55.186295 28952 ts_tablet_manager.cc:933] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:46:55.203063 28952 tablet_copy_client.cc:323] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.2:34855
I20260807 01:46:55.206444 27250 tablet_copy_service.cc:140] P 3478729505b046c2a0051c2d0130873f: Received BeginTabletCopySession request for tablet 94962ef95ca14973bc39a17ed76c0087 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:41230)
I20260807 01:46:55.207055 27250 tablet_copy_service.cc:161] P 3478729505b046c2a0051c2d0130873f: Beginning new tablet copy session on tablet 94962ef95ca14973bc39a17ed76c0087 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:41230: session id = 57bb4985157447f0a3ab680fa0026f12-94962ef95ca14973bc39a17ed76c0087
I20260807 01:46:55.222874 27250 tablet_copy_source_session.cc:215] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:46:55.227176 28952 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94962ef95ca14973bc39a17ed76c0087. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:55.235634 28715 raft_consensus.cc:1217] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20260807 01:46:55.263648 28930 tablet_bootstrap.cc:492] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:46:55.272284 28930 tablet_bootstrap.cc:492] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:46:55.273240 28930 ts_tablet_manager.cc:1403] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.151s user 0.030s sys 0.000s
I20260807 01:46:55.277551 28930 raft_consensus.cc:359] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:55.273609 28952 tablet_copy_client.cc:806] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:46:55.279102 28952 tablet_copy_client.cc:670] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:46:55.284372 28930 raft_consensus.cc:740] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:46:55.286844 28930 consensus_queue.cc:260] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:55.301733 28952 tablet_copy_client.cc:538] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:46:55.316906 28930 ts_tablet_manager.cc:1434] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.043s user 0.006s sys 0.002s
I20260807 01:46:55.324518 27250 tablet_copy_service.cc:342] P 3478729505b046c2a0051c2d0130873f: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-8767bf1fc98c4effa8f3b5f5113afe40 received from {username='slave'} at 127.0.0.1:41230
I20260807 01:46:55.325044 27250 tablet_copy_service.cc:434] P 3478729505b046c2a0051c2d0130873f: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-8767bf1fc98c4effa8f3b5f5113afe40 on tablet 8767bf1fc98c4effa8f3b5f5113afe40 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:46:55.330741 28930 ts_tablet_manager.cc:933] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:55.354101 28952 tablet_bootstrap.cc:492] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:46:55.389392 28930 tablet_copy_client.cc:323] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.3:42429
I20260807 01:46:55.393316 27350 tablet_copy_service.cc:140] P fcf4e45defb94276bcfa68392711dade: Received BeginTabletCopySession request for tablet 2cfefe1ff8f14fb68678ac128e129d32 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:36604)
I20260807 01:46:55.393930 27350 tablet_copy_service.cc:161] P fcf4e45defb94276bcfa68392711dade: Beginning new tablet copy session on tablet 2cfefe1ff8f14fb68678ac128e129d32 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:36604: session id = 57bb4985157447f0a3ab680fa0026f12-2cfefe1ff8f14fb68678ac128e129d32
I20260807 01:46:55.412043 27350 tablet_copy_source_session.cc:215] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:46:55.463502 28930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cfefe1ff8f14fb68678ac128e129d32. 1 dirs total, 0 dirs full, 0 dirs failed
W20260807 01:46:55.478788 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 58653.3us
I20260807 01:46:55.526595 28952 tablet_bootstrap.cc:492] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:46:55.527786 28952 tablet_bootstrap.cc:492] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:46:55.532975 28952 ts_tablet_manager.cc:1403] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.179s user 0.020s sys 0.012s
I20260807 01:46:55.548907 28952 raft_consensus.cc:359] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:55.550038 28952 raft_consensus.cc:740] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:46:55.551280 28952 consensus_queue.cc:260] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:55.554275 28930 tablet_copy_client.cc:806] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:46:55.555540 28930 tablet_copy_client.cc:670] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:46:55.561038 28952 ts_tablet_manager.cc:1434] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.028s user 0.007s sys 0.000s
I20260807 01:46:55.567387 27250 tablet_copy_service.cc:342] P 3478729505b046c2a0051c2d0130873f: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-94962ef95ca14973bc39a17ed76c0087 received from {username='slave'} at 127.0.0.1:41230
I20260807 01:46:55.567988 27250 tablet_copy_service.cc:434] P 3478729505b046c2a0051c2d0130873f: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-94962ef95ca14973bc39a17ed76c0087 on tablet 94962ef95ca14973bc39a17ed76c0087 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:46:55.584115 28930 tablet_copy_client.cc:538] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:46:55.601930 28715 raft_consensus.cc:1217] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260807 01:46:55.616534 28930 tablet_bootstrap.cc:492] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:46:55.658361 28711 raft_consensus.cc:1217] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.3->[4.4-4.4] Dedup: 4.4->[]
I20260807 01:46:55.726992 28908 raft_consensus.cc:1064] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:46:55.729954 28908 consensus_queue.cc:237] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:46:55.822356 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 44430.5us
I20260807 01:46:55.822364 28930 tablet_bootstrap.cc:492] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:46:55.839404 28930 tablet_bootstrap.cc:492] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:46:55.840193 28930 ts_tablet_manager.cc:1403] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.224s user 0.017s sys 0.020s
I20260807 01:46:55.843531 28711 raft_consensus.cc:1275] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 01:46:55.847059 27338 raft_consensus.cc:1275] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 01:46:55.852771 28908 consensus_queue.cc:1048] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:46:55.859655 28973 consensus_queue.cc:1048] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:46:55.872941 27227 raft_consensus.cc:1275] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 01:46:55.885468 28972 consensus_queue.cc:1048] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:46:55.849642 28930 raft_consensus.cc:359] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:55.891512 28930 raft_consensus.cc:740] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:46:55.892282 28930 consensus_queue.cc:260] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:55.905566 28930 ts_tablet_manager.cc:1434] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.062s user 0.005s sys 0.001s
I20260807 01:46:55.923995 28972 raft_consensus.cc:2955] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:55.931298 27227 raft_consensus.cc:2955] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:55.932917 27350 tablet_copy_service.cc:342] P fcf4e45defb94276bcfa68392711dade: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-2cfefe1ff8f14fb68678ac128e129d32 received from {username='slave'} at 127.0.0.1:36604
I20260807 01:46:55.935012 27350 tablet_copy_service.cc:434] P fcf4e45defb94276bcfa68392711dade: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-2cfefe1ff8f14fb68678ac128e129d32 on tablet 2cfefe1ff8f14fb68678ac128e129d32 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:46:55.942433 28711 raft_consensus.cc:2955] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:55.962544 26950 catalog_manager.cc:5697] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:55.983863 27333 raft_consensus.cc:2955] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:56.044792 28918 raft_consensus.cc:1064] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:46:56.050447 28918 consensus_queue.cc:237] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:46:56.085536 28711 raft_consensus.cc:1275] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:46:56.087673 28446 consensus_queue.cc:1048] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:46:56.143554 27105 raft_consensus.cc:1275] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:46:56.152421 28919 consensus_queue.cc:1048] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:46:56.159299 27330 raft_consensus.cc:1275] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:46:56.171087 28919 consensus_queue.cc:1048] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:46:56.212175 27099 consensus_queue.cc:237] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:46:56.221751 28711 raft_consensus.cc:1275] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:46:56.224211 28917 consensus_queue.cc:1048] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:46:56.226182 28918 raft_consensus.cc:2955] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:56.233078 28917 raft_consensus.cc:2955] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:56.233568 27105 raft_consensus.cc:2955] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:56.237442 28715 raft_consensus.cc:2955] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:56.245950 27331 raft_consensus.cc:2955] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:56.274941 28446 raft_consensus.cc:1064] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:46:56.277467 28446 consensus_queue.cc:237] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 2, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:46:56.287528 27333 raft_consensus.cc:1275] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:46:56.286070 26949 catalog_manager.cc:5697] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:46:56.289511 28711 raft_consensus.cc:2955] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:56.302937 28972 consensus_queue.cc:1048] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:46:56.322232 27333 raft_consensus.cc:2955] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:56.345033 26950 catalog_manager.cc:5697] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 4 to 5, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New cstate: current_term: 2 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:56.380865 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 51cef821f0284d8e9369e98d38f80e73 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:46:56.405377 28711 raft_consensus.cc:1275] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 01:46:56.407359 28898 consensus_queue.cc:1048] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:46:56.430174 27330 raft_consensus.cc:1275] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 01:46:56.438747 27227 consensus_queue.cc:237] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:46:56.439011 29002 consensus_queue.cc:1048] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:46:56.446538 27105 raft_consensus.cc:1275] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 01:46:56.455282 27199 tablet_service.cc:1558] Processing DeleteTablet for tablet 51cef821f0284d8e9369e98d38f80e73 with delete_type TABLET_DATA_TOMBSTONED (TS 3478729505b046c2a0051c2d0130873f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:53780
I20260807 01:46:56.473763 29006 tablet_replica.cc:333] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f: stopping tablet replica
I20260807 01:46:56.484876 28715 raft_consensus.cc:1217] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2] Dedup: 2.2->[]
I20260807 01:46:56.494341 28446 consensus_queue.cc:1048] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:46:56.496008 28715 raft_consensus.cc:1275] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:46:56.497766 28919 consensus_queue.cc:1048] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:46:56.499598 27109 raft_consensus.cc:1275] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:46:56.502058 28996 consensus_queue.cc:1048] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:46:56.525655 28919 raft_consensus.cc:2955] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:56.526201 28715 raft_consensus.cc:2955] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
W20260807 01:46:56.531239 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 45155.2us
I20260807 01:46:56.533875 28898 raft_consensus.cc:2955] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:56.536893 27099 raft_consensus.cc:2955] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:56.556407 27109 raft_consensus.cc:2955] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:56.567674 27331 raft_consensus.cc:2955] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:56.632354 28711 raft_consensus.cc:2955] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:56.649680 26949 catalog_manager.cc:5697] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:46:56.702533 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 94962ef95ca14973bc39a17ed76c0087 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:46:56.703141 26950 catalog_manager.cc:5697] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 3 to 4, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:56.716898 29006 raft_consensus.cc:2243] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:56.716775 27227 consensus_queue.cc:237] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:46:56.717839 29006 raft_consensus.cc:2272] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:56.732437 27317 tablet_service.cc:1558] Processing DeleteTablet for tablet 94962ef95ca14973bc39a17ed76c0087 with delete_type TABLET_DATA_TOMBSTONED (TS fcf4e45defb94276bcfa68392711dade not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:59474
I20260807 01:46:56.732744 29006 ts_tablet_manager.cc:1916] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:46:56.767148 28711 raft_consensus.cc:1275] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 01:46:56.770772 28446 consensus_queue.cc:1048] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 01:46:56.780129 28996 raft_consensus.cc:2955] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:56.783938 29014 tablet_replica.cc:333] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade: stopping tablet replica
I20260807 01:46:56.785843 29014 raft_consensus.cc:2243] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:56.786983 29014 raft_consensus.cc:2272] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:56.788179 27101 raft_consensus.cc:1275] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 01:46:56.790392 28715 raft_consensus.cc:2955] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:56.794284 29014 ts_tablet_manager.cc:1916] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:46:56.808022 28446 consensus_queue.cc:1048] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:46:56.811671 29006 ts_tablet_manager.cc:1929] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260807 01:46:56.816203 29006 log.cc:1199] T 51cef821f0284d8e9369e98d38f80e73 P 3478729505b046c2a0051c2d0130873f: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-1-root/wals/51cef821f0284d8e9369e98d38f80e73
I20260807 01:46:56.834573 29014 ts_tablet_manager.cc:1929] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 01:46:56.836127 29014 log.cc:1199] T 94962ef95ca14973bc39a17ed76c0087 P fcf4e45defb94276bcfa68392711dade: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-2-root/wals/94962ef95ca14973bc39a17ed76c0087
I20260807 01:46:56.832741 26932 catalog_manager.cc:5028] TS 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): tablet 51cef821f0284d8e9369e98d38f80e73 (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:46:56.838558 26931 catalog_manager.cc:5028] TS fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): tablet 94962ef95ca14973bc39a17ed76c0087 (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:46:56.849457 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8767bf1fc98c4effa8f3b5f5113afe40 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:46:56.850069 26949 catalog_manager.cc:5697] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 5 to 6, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New cstate: current_term: 4 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:56.853197 27101 raft_consensus.cc:2955] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:56.872366 27317 tablet_service.cc:1558] Processing DeleteTablet for tablet 8767bf1fc98c4effa8f3b5f5113afe40 with delete_type TABLET_DATA_TOMBSTONED (TS fcf4e45defb94276bcfa68392711dade not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:59474
I20260807 01:46:56.873410 29014 tablet_replica.cc:333] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade: stopping tablet replica
I20260807 01:46:56.874519 29014 raft_consensus.cc:2243] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:56.875190 29014 raft_consensus.cc:2272] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:56.879230 29014 ts_tablet_manager.cc:1916] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:46:56.896885 29014 ts_tablet_manager.cc:1929] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.5
I20260807 01:46:56.897353 29014 log.cc:1199] T 8767bf1fc98c4effa8f3b5f5113afe40 P fcf4e45defb94276bcfa68392711dade: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-2-root/wals/8767bf1fc98c4effa8f3b5f5113afe40
I20260807 01:46:56.900002 26931 catalog_manager.cc:5028] TS fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): tablet 8767bf1fc98c4effa8f3b5f5113afe40 (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:46:57.103674 28969 raft_consensus.cc:1064] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:46:57.107780 28969 consensus_queue.cc:237] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:46:57.118449 28715 raft_consensus.cc:1275] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 01:46:57.121498 27229 raft_consensus.cc:1275] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 01:46:57.121855 27101 raft_consensus.cc:1275] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 01:46:57.122377 28969 consensus_queue.cc:1048] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260807 01:46:57.124866 29011 consensus_queue.cc:1048] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:46:57.126875 28968 consensus_queue.cc:1048] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:46:57.148330 28968 raft_consensus.cc:2955] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:57.150970 27229 raft_consensus.cc:2955] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:57.159129 27101 raft_consensus.cc:2955] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:57.160874 28715 raft_consensus.cc:2955] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:57.167716 26948 catalog_manager.cc:5697] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:46:57.204484 27328 consensus_queue.cc:237] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:46:57.210557 28715 raft_consensus.cc:1275] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 01:46:57.212131 29011 consensus_queue.cc:1048] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:46:57.226534 27229 raft_consensus.cc:1275] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 01:46:57.237876 29011 consensus_queue.cc:1048] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:46:57.252434 29010 raft_consensus.cc:2955] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:57.268393 27229 raft_consensus.cc:2955] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:57.272217 26931 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2cfefe1ff8f14fb68678ac128e129d32 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:46:57.283385 28715 raft_consensus.cc:2955] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:57.286499 26950 catalog_manager.cc:5697] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New cstate: current_term: 2 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:57.326030 27088 tablet_service.cc:1558] Processing DeleteTablet for tablet 2cfefe1ff8f14fb68678ac128e129d32 with delete_type TABLET_DATA_TOMBSTONED (TS 3262a9d183914b699227fdba0ca5bed3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41078
I20260807 01:46:57.346735 29021 tablet_replica.cc:333] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3: stopping tablet replica
I20260807 01:46:57.348021 29021 raft_consensus.cc:2243] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:46:57.349428 29021 raft_consensus.cc:2272] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:46:57.352939 29021 ts_tablet_manager.cc:1916] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:46:57.366704 29021 ts_tablet_manager.cc:1929] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260807 01:46:57.367144 29021 log.cc:1199] T 2cfefe1ff8f14fb68678ac128e129d32 P 3262a9d183914b699227fdba0ca5bed3: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-0-root/wals/2cfefe1ff8f14fb68678ac128e129d32
I20260807 01:46:57.368642 26932 catalog_manager.cc:5028] TS 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): tablet 2cfefe1ff8f14fb68678ac128e129d32 (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:46:58.413909 27105 consensus_queue.cc:237] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:58.421481 27229 raft_consensus.cc:1275] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260807 01:46:58.421864 27067 consensus_peers.cc:597] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c8799906440c4ef19cfbadc18cf32ef0. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:58.422978 27333 raft_consensus.cc:1275] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 01:46:58.424966 29023 consensus_queue.cc:1048] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:58.426033 28978 consensus_queue.cc:1048] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:58.434569 28978 raft_consensus.cc:2955] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:58.439124 27333 raft_consensus.cc:2955] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:58.444818 27229 raft_consensus.cc:2955] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:58.455103 27105 consensus_queue.cc:237] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:58.461092 26952 catalog_manager.cc:5697] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:58.476183 27333 raft_consensus.cc:1275] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 01:46:58.477491 27229 raft_consensus.cc:1275] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260807 01:46:58.479529 27067 consensus_peers.cc:597] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 879666f362ba4a2db3fdfe6738b37dec. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:58.479175 28978 consensus_queue.cc:1048] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:58.481426 28916 consensus_queue.cc:1048] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:58.495524 28978 raft_consensus.cc:2955] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:58.499925 27333 raft_consensus.cc:2955] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:58.504349 27227 raft_consensus.cc:2955] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:58.519748 27105 consensus_queue.cc:237] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:58.528484 26949 catalog_manager.cc:5697] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:58.538203 27333 raft_consensus.cc:1275] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260807 01:46:58.540020 27067 consensus_peers.cc:597] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f27eab393b1f4f21aad32707749effa3. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:58.542062 29023 consensus_queue.cc:1048] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:58.542608 27227 raft_consensus.cc:1275] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 01:46:58.547813 28916 consensus_queue.cc:1048] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:46:58.556564 29023 raft_consensus.cc:2955] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:58.559245 27333 raft_consensus.cc:2955] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:58.561635 27227 raft_consensus.cc:2955] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:58.581447 27328 consensus_queue.cc:237] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:58.583496 26952 catalog_manager.cc:5697] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
W20260807 01:46:58.606328 27289 consensus_peers.cc:597] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b2dac5b83a404fbba1506670f8b8b45f. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:46:58.609225 27227 raft_consensus.cc:1275] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 01:46:58.609547 27105 raft_consensus.cc:1275] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 01:46:58.610687 29010 consensus_queue.cc:1048] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:58.615202 29010 consensus_queue.cc:1048] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:46:58.619518 28969 raft_consensus.cc:2955] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:58.621206 27227 raft_consensus.cc:2955] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:58.628001 27105 raft_consensus.cc:2955] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:58.647038 26949 catalog_manager.cc:5697] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:46:58.954615 29053 ts_tablet_manager.cc:933] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:46:58.956542 29053 tablet_copy_client.cc:323] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.1:35499
I20260807 01:46:58.958247 27127 tablet_copy_service.cc:140] P 3262a9d183914b699227fdba0ca5bed3: Received BeginTabletCopySession request for tablet 879666f362ba4a2db3fdfe6738b37dec from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:52556)
I20260807 01:46:58.958762 27127 tablet_copy_service.cc:161] P 3262a9d183914b699227fdba0ca5bed3: Beginning new tablet copy session on tablet 879666f362ba4a2db3fdfe6738b37dec from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:52556: session id = 57bb4985157447f0a3ab680fa0026f12-879666f362ba4a2db3fdfe6738b37dec
I20260807 01:46:58.968119 27127 tablet_copy_source_session.cc:215] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:46:58.971796 29053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 879666f362ba4a2db3fdfe6738b37dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:58.987020 29053 tablet_copy_client.cc:806] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:46:58.987700 29053 tablet_copy_client.cc:670] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:46:58.992458 29053 tablet_copy_client.cc:538] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:46:59.002012 29053 tablet_bootstrap.cc:492] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:46:59.022035 29057 ts_tablet_manager.cc:933] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:46:59.027190 29053 tablet_bootstrap.cc:492] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:46:59.028455 29053 tablet_bootstrap.cc:492] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:46:59.028604 29057 tablet_copy_client.cc:323] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.1:35499
I20260807 01:46:59.029186 29053 ts_tablet_manager.cc:1403] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.027s user 0.018s sys 0.011s
I20260807 01:46:59.031262 27127 tablet_copy_service.cc:140] P 3262a9d183914b699227fdba0ca5bed3: Received BeginTabletCopySession request for tablet c8799906440c4ef19cfbadc18cf32ef0 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:52556)
I20260807 01:46:59.031872 27127 tablet_copy_service.cc:161] P 3262a9d183914b699227fdba0ca5bed3: Beginning new tablet copy session on tablet c8799906440c4ef19cfbadc18cf32ef0 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:52556: session id = 57bb4985157447f0a3ab680fa0026f12-c8799906440c4ef19cfbadc18cf32ef0
I20260807 01:46:59.032167 29053 raft_consensus.cc:359] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:59.032776 29053 raft_consensus.cc:740] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:46:59.033301 29053 consensus_queue.cc:260] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:59.040544 27127 tablet_copy_source_session.cc:215] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:46:59.041793 29053 ts_tablet_manager.cc:1434] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.012s user 0.006s sys 0.004s
I20260807 01:46:59.044994 27126 tablet_copy_service.cc:342] P 3262a9d183914b699227fdba0ca5bed3: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-879666f362ba4a2db3fdfe6738b37dec received from {username='slave'} at 127.0.0.1:52556
I20260807 01:46:59.045452 27126 tablet_copy_service.cc:434] P 3262a9d183914b699227fdba0ca5bed3: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-879666f362ba4a2db3fdfe6738b37dec on tablet 879666f362ba4a2db3fdfe6738b37dec with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:46:59.055433 29053 ts_tablet_manager.cc:933] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:46:59.063349 29057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8799906440c4ef19cfbadc18cf32ef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:59.086455 29053 tablet_copy_client.cc:323] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.3:42429
I20260807 01:46:59.088869 27350 tablet_copy_service.cc:140] P fcf4e45defb94276bcfa68392711dade: Received BeginTabletCopySession request for tablet b2dac5b83a404fbba1506670f8b8b45f from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:36604)
I20260807 01:46:59.089650 27350 tablet_copy_service.cc:161] P fcf4e45defb94276bcfa68392711dade: Beginning new tablet copy session on tablet b2dac5b83a404fbba1506670f8b8b45f from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:36604: session id = 57bb4985157447f0a3ab680fa0026f12-b2dac5b83a404fbba1506670f8b8b45f
I20260807 01:46:59.100085 27350 tablet_copy_source_session.cc:215] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:46:59.100850 29057 tablet_copy_client.cc:806] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:46:59.101584 29057 tablet_copy_client.cc:670] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:46:59.103492 29053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2dac5b83a404fbba1506670f8b8b45f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:59.113008 29057 tablet_copy_client.cc:538] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:46:59.122460 29057 tablet_bootstrap.cc:492] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:46:59.158380 29061 ts_tablet_manager.cc:933] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:46:59.173393 29057 tablet_bootstrap.cc:492] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:46:59.174676 29057 tablet_bootstrap.cc:492] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:46:59.175478 29057 ts_tablet_manager.cc:1403] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.053s user 0.035s sys 0.012s
I20260807 01:46:59.178892 29057 raft_consensus.cc:359] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:59.180217 29057 raft_consensus.cc:740] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:46:59.180998 29057 consensus_queue.cc:260] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:59.186707 29057 ts_tablet_manager.cc:1434] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.011s user 0.004s sys 0.003s
I20260807 01:46:59.188593 27127 tablet_copy_service.cc:342] P 3262a9d183914b699227fdba0ca5bed3: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-c8799906440c4ef19cfbadc18cf32ef0 received from {username='slave'} at 127.0.0.1:52556
I20260807 01:46:59.189050 27127 tablet_copy_service.cc:434] P 3262a9d183914b699227fdba0ca5bed3: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-c8799906440c4ef19cfbadc18cf32ef0 on tablet c8799906440c4ef19cfbadc18cf32ef0 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:46:59.197383 29061 tablet_copy_client.cc:323] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.1:35499
I20260807 01:46:59.197762 29053 tablet_copy_client.cc:806] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:46:59.199972 27127 tablet_copy_service.cc:140] P 3262a9d183914b699227fdba0ca5bed3: Received BeginTabletCopySession request for tablet f27eab393b1f4f21aad32707749effa3 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:52556)
I20260807 01:46:59.200546 27127 tablet_copy_service.cc:161] P 3262a9d183914b699227fdba0ca5bed3: Beginning new tablet copy session on tablet f27eab393b1f4f21aad32707749effa3 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:52556: session id = 57bb4985157447f0a3ab680fa0026f12-f27eab393b1f4f21aad32707749effa3
I20260807 01:46:59.207695 29053 tablet_copy_client.cc:670] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:46:59.212841 29053 tablet_copy_client.cc:538] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:46:59.216735 27127 tablet_copy_source_session.cc:215] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:46:59.220182 29061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f27eab393b1f4f21aad32707749effa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:46:59.222841 29053 tablet_bootstrap.cc:492] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:46:59.242282 29061 tablet_copy_client.cc:806] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:46:59.243650 29061 tablet_copy_client.cc:670] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:46:59.249270 29061 tablet_copy_client.cc:538] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:46:59.252735 29053 tablet_bootstrap.cc:492] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:46:59.254483 29053 tablet_bootstrap.cc:492] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:46:59.255894 29053 ts_tablet_manager.cc:1403] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.033s user 0.026s sys 0.004s
I20260807 01:46:59.264024 29053 raft_consensus.cc:359] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:59.264963 29053 raft_consensus.cc:740] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:46:59.265686 29053 consensus_queue.cc:260] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:59.272552 29061 tablet_bootstrap.cc:492] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:46:59.273147 29053 ts_tablet_manager.cc:1434] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.017s user 0.005s sys 0.004s
I20260807 01:46:59.281584 27350 tablet_copy_service.cc:342] P fcf4e45defb94276bcfa68392711dade: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-b2dac5b83a404fbba1506670f8b8b45f received from {username='slave'} at 127.0.0.1:36604
I20260807 01:46:59.282347 27350 tablet_copy_service.cc:434] P fcf4e45defb94276bcfa68392711dade: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-b2dac5b83a404fbba1506670f8b8b45f on tablet b2dac5b83a404fbba1506670f8b8b45f with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:46:59.353067 29061 tablet_bootstrap.cc:492] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:46:59.354853 29061 tablet_bootstrap.cc:492] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:46:59.359668 29061 ts_tablet_manager.cc:1403] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.088s user 0.028s sys 0.039s
I20260807 01:46:59.368064 29061 raft_consensus.cc:359] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:59.369551 29061 raft_consensus.cc:740] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:46:59.370875 29061 consensus_queue.cc:260] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:46:59.375751 29061 ts_tablet_manager.cc:1434] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.015s user 0.008s sys 0.008s
I20260807 01:46:59.380581 27127 tablet_copy_service.cc:342] P 3262a9d183914b699227fdba0ca5bed3: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-f27eab393b1f4f21aad32707749effa3 received from {username='slave'} at 127.0.0.1:52556
I20260807 01:46:59.381057 27127 tablet_copy_service.cc:434] P 3262a9d183914b699227fdba0ca5bed3: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-f27eab393b1f4f21aad32707749effa3 on tablet f27eab393b1f4f21aad32707749effa3 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:46:59.445019 28711 raft_consensus.cc:1217] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260807 01:46:59.490267 28711 raft_consensus.cc:1217] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260807 01:46:59.496013 28711 raft_consensus.cc:1217] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260807 01:46:59.528226 28711 raft_consensus.cc:1217] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260807 01:46:59.841893 29023 raft_consensus.cc:1064] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:46:59.844733 29023 consensus_queue.cc:237] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:46:59.854288 27229 raft_consensus.cc:1275] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:46:59.856396 28916 consensus_queue.cc:1048] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:46:59.856962 27333 raft_consensus.cc:1275] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:46:59.861189 28916 consensus_queue.cc:1048] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:46:59.865108 28711 raft_consensus.cc:1275] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:46:59.869536 29079 consensus_queue.cc:1048] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:46:59.877732 29074 raft_consensus.cc:2955] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:59.885425 27229 raft_consensus.cc:2955] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:59.906544 27338 raft_consensus.cc:2955] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:59.913378 26949 catalog_manager.cc:5697] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 01:46:59.940361 28715 raft_consensus.cc:2955] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:46:59.973868 29023 raft_consensus.cc:1064] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:46:59.976682 29023 consensus_queue.cc:237] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:46:59.999372 27109 consensus_queue.cc:237] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:00.006379 28715 raft_consensus.cc:1275] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:47:00.009896 27338 raft_consensus.cc:1275] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:47:00.034498 29074 raft_consensus.cc:1064] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:00.052398 28978 consensus_queue.cc:1048] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:00.055402 29074 consensus_queue.cc:237] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:00.061372 28916 consensus_queue.cc:1048] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:00.069456 29067 raft_consensus.cc:1064] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:00.086234 28711 raft_consensus.cc:1275] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:47:00.086431 29067 consensus_queue.cc:237] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:00.089591 29091 consensus_queue.cc:1048] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:00.091573 28711 raft_consensus.cc:2955] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:00.095177 27338 raft_consensus.cc:1275] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:47:00.108592 27229 raft_consensus.cc:1275] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:47:00.110545 29023 consensus_queue.cc:1048] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:00.098747 28978 raft_consensus.cc:2955] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:00.119935 28916 consensus_queue.cc:1048] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:00.132537 27330 raft_consensus.cc:2955] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:00.150422 28714 raft_consensus.cc:1275] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:47:00.153198 29090 consensus_queue.cc:1048] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:00.192142 27109 raft_consensus.cc:1275] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:47:00.193533 27229 raft_consensus.cc:1275] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:47:00.197577 29105 consensus_queue.cc:1048] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:00.205605 26952 catalog_manager.cc:5697] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 3 to 4, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:00.212605 29103 consensus_queue.cc:1048] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:00.214354 29091 raft_consensus.cc:2955] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:00.220392 27199 tablet_service.cc:1558] Processing DeleteTablet for tablet c8799906440c4ef19cfbadc18cf32ef0 with delete_type TABLET_DATA_TOMBSTONED (TS 3478729505b046c2a0051c2d0130873f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53780
W20260807 01:47:00.225430 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 73408.3us
I20260807 01:47:00.236174 27331 raft_consensus.cc:2955] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:00.275699 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c8799906440c4ef19cfbadc18cf32ef0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:00.304834 29109 tablet_replica.cc:333] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f: stopping tablet replica
I20260807 01:47:00.315500 29109 raft_consensus.cc:2243] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:00.311301 28711 raft_consensus.cc:2955] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:00.322419 28714 raft_consensus.cc:1275] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:47:00.329448 27212 raft_consensus.cc:1275] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:47:00.349504 29079 consensus_queue.cc:1048] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260807 01:47:00.359459 29109 raft_consensus.cc:2272] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:00.367210 29074 consensus_queue.cc:1048] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:00.388438 29109 ts_tablet_manager.cc:1916] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260807 01:47:00.401809 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 59495.4us
I20260807 01:47:00.449344 27229 raft_consensus.cc:2955] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:00.467437 29067 raft_consensus.cc:2955] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:00.475690 28715 raft_consensus.cc:2955] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
W20260807 01:47:00.483196 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 118922us
I20260807 01:47:00.514611 26952 catalog_manager.cc:5697] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:00.519726 27330 raft_consensus.cc:1275] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:47:00.597770 26949 catalog_manager.cc:5697] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:00.612166 27101 consensus_queue.cc:237] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:00.632375 28715 raft_consensus.cc:1275] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:47:00.634907 29023 consensus_queue.cc:1048] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:00.656594 27229 raft_consensus.cc:2955] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:00.681253 29023 raft_consensus.cc:2955] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:00.697330 28715 raft_consensus.cc:2955] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:00.701661 27101 raft_consensus.cc:2955] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
W20260807 01:47:00.717602 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 256972us
I20260807 01:47:00.783960 26949 catalog_manager.cc:5697] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 3 to 4, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:00.807029 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f27eab393b1f4f21aad32707749effa3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:00.830207 27333 raft_consensus.cc:1275] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:47:00.859702 29079 consensus_queue.cc:1048] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:00.889019 29109 ts_tablet_manager.cc:1929] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 01:47:00.889696 29109 log.cc:1199] T c8799906440c4ef19cfbadc18cf32ef0 P 3478729505b046c2a0051c2d0130873f: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-1-root/wals/c8799906440c4ef19cfbadc18cf32ef0
W20260807 01:47:00.891496 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 86469.5us
I20260807 01:47:00.894577 29023 consensus_queue.cc:1048] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:00.902989 29123 raft_consensus.cc:2955] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:00.923977 27199 tablet_service.cc:1558] Processing DeleteTablet for tablet f27eab393b1f4f21aad32707749effa3 with delete_type TABLET_DATA_TOMBSTONED (TS 3478729505b046c2a0051c2d0130873f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53780
I20260807 01:47:00.935921 28715 raft_consensus.cc:2955] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:00.938136 27338 consensus_queue.cc:237] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:00.948091 28713 raft_consensus.cc:1275] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:47:00.951462 29067 consensus_queue.cc:1048] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:00.964594 27230 raft_consensus.cc:2955] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:00.974439 29104 raft_consensus.cc:2955] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:00.976351 28714 raft_consensus.cc:2955] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:00.979960 29109 tablet_replica.cc:333] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f: stopping tablet replica
I20260807 01:47:00.979998 26932 catalog_manager.cc:5028] TS 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): tablet c8799906440c4ef19cfbadc18cf32ef0 (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:00.991489 27338 raft_consensus.cc:2955] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:01.012132 26949 catalog_manager.cc:5697] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:00.981940 29109 raft_consensus.cc:2243] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:01.015447 29109 raft_consensus.cc:2272] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:01.025053 27230 raft_consensus.cc:1275] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:47:01.084685 26931 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b2dac5b83a404fbba1506670f8b8b45f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:01.089115 29109 ts_tablet_manager.cc:1916] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:01.081225 27103 consensus_queue.cc:237] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:01.093546 26948 catalog_manager.cc:5697] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New cstate: current_term: 1 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:01.106837 28714 raft_consensus.cc:1275] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:47:01.109886 29126 consensus_queue.cc:1048] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260807 01:47:01.119423 29109 ts_tablet_manager.cc:1929] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 01:47:01.120071 29109 log.cc:1199] T f27eab393b1f4f21aad32707749effa3 P 3478729505b046c2a0051c2d0130873f: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-1-root/wals/f27eab393b1f4f21aad32707749effa3
I20260807 01:47:01.121197 27230 raft_consensus.cc:1275] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:47:01.122409 26932 catalog_manager.cc:5028] TS 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): tablet f27eab393b1f4f21aad32707749effa3 (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:01.136023 29104 consensus_queue.cc:1048] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:01.140683 27088 tablet_service.cc:1558] Processing DeleteTablet for tablet b2dac5b83a404fbba1506670f8b8b45f with delete_type TABLET_DATA_TOMBSTONED (TS 3262a9d183914b699227fdba0ca5bed3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41078
I20260807 01:47:01.142779 28714 raft_consensus.cc:2955] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:01.150696 29123 consensus_queue.cc:1048] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:01.155321 27229 raft_consensus.cc:2955] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:01.179725 29126 raft_consensus.cc:2955] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:01.188359 29131 tablet_replica.cc:333] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3: stopping tablet replica
I20260807 01:47:01.190765 29131 raft_consensus.cc:2243] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:01.192189 29131 raft_consensus.cc:2272] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:01.198469 27230 raft_consensus.cc:2955] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:01.200291 29131 ts_tablet_manager.cc:1916] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:01.226768 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 879666f362ba4a2db3fdfe6738b37dec with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:01.239080 29131 ts_tablet_manager.cc:1929] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 01:47:01.240163 29131 log.cc:1199] T b2dac5b83a404fbba1506670f8b8b45f P 3262a9d183914b699227fdba0ca5bed3: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-0-root/wals/b2dac5b83a404fbba1506670f8b8b45f
I20260807 01:47:01.243695 26932 catalog_manager.cc:5028] TS 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): tablet b2dac5b83a404fbba1506670f8b8b45f (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:01.241348 26948 catalog_manager.cc:5697] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 3 to 4, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:01.276787 27317 tablet_service.cc:1558] Processing DeleteTablet for tablet 879666f362ba4a2db3fdfe6738b37dec with delete_type TABLET_DATA_TOMBSTONED (TS fcf4e45defb94276bcfa68392711dade not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:59474
I20260807 01:47:01.280836 29134 tablet_replica.cc:333] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade: stopping tablet replica
I20260807 01:47:01.285321 29134 raft_consensus.cc:2243] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:01.286293 29134 pending_rounds.cc:70] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade: Trying to abort 1 pending ops.
I20260807 01:47:01.286741 29134 pending_rounds.cc:77] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7315731333329129472 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "879666f362ba4a2db3fdfe6738b37dec" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } } }
I20260807 01:47:01.289238 29134 raft_consensus.cc:2917] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20260807 01:47:01.289811 29134 raft_consensus.cc:2889] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260807 01:47:01.290387 29134 raft_consensus.cc:2272] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:01.302644 29134 ts_tablet_manager.cc:1916] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:01.345155 29134 ts_tablet_manager.cc:1929] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 01:47:01.345630 29134 log.cc:1199] T 879666f362ba4a2db3fdfe6738b37dec P fcf4e45defb94276bcfa68392711dade: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-2-root/wals/879666f362ba4a2db3fdfe6738b37dec
I20260807 01:47:01.347433 26931 catalog_manager.cc:5028] TS fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): tablet 879666f362ba4a2db3fdfe6738b37dec (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:02.364537 27328 consensus_queue.cc:237] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
W20260807 01:47:02.371176 27289 consensus_peers.cc:597] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2c3fdef784cc4afbb8dda87287a7eac0. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:02.371417 27230 raft_consensus.cc:1275] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:02.372519 27102 raft_consensus.cc:1275] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:02.372850 29141 consensus_queue.cc:1048] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:47:02.374531 29101 consensus_queue.cc:1048] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:47:02.387074 29141 raft_consensus.cc:2955] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:02.393292 27230 raft_consensus.cc:2955] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:02.404569 27102 raft_consensus.cc:2955] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:02.416998 27230 consensus_queue.cc:237] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:02.430124 26949 catalog_manager.cc:5697] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 2 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260807 01:47:02.441346 27175 consensus_peers.cc:597] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60a03ab231464be89d6cce9e6988cfba. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:02.445004 27333 raft_consensus.cc:1275] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 01:47:02.452755 29150 consensus_queue.cc:1048] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:02.456245 27102 raft_consensus.cc:1275] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 01:47:02.457782 29113 consensus_queue.cc:1048] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:02.463644 29150 raft_consensus.cc:2955] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:02.465376 27333 raft_consensus.cc:2955] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:02.484862 27230 consensus_queue.cc:237] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:02.491443 26950 catalog_manager.cc:5697] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:02.501562 27102 raft_consensus.cc:2955] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
W20260807 01:47:02.504122 27175 consensus_peers.cc:597] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 498dfe25861a4b4fbf2aca930b357274. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:02.506878 27333 raft_consensus.cc:1275] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 01:47:02.511567 29113 consensus_queue.cc:1048] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:02.520861 27102 raft_consensus.cc:1275] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 01:47:02.523015 29159 consensus_queue.cc:1048] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:02.525892 29150 raft_consensus.cc:2955] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:02.529758 27333 raft_consensus.cc:2955] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:02.547834 26950 catalog_manager.cc:5697] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:02.548666 27230 consensus_queue.cc:237] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:02.551826 27101 raft_consensus.cc:2955] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
W20260807 01:47:02.573524 27175 consensus_peers.cc:597] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bf07cc968f3343869d12c10e81ce75f9. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:02.577947 27102 raft_consensus.cc:1275] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 01:47:02.580629 29113 consensus_queue.cc:1048] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:47:02.581835 27328 raft_consensus.cc:1275] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 01:47:02.585875 29158 consensus_queue.cc:1048] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:47:02.589854 29113 raft_consensus.cc:2955] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:02.592309 27102 raft_consensus.cc:2955] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:02.601365 27333 raft_consensus.cc:2955] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:02.625700 26950 catalog_manager.cc:5697] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:02.872764 29170 ts_tablet_manager.cc:933] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:02.874769 29170 tablet_copy_client.cc:323] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.3:42429
I20260807 01:47:02.876425 27350 tablet_copy_service.cc:140] P fcf4e45defb94276bcfa68392711dade: Received BeginTabletCopySession request for tablet 2c3fdef784cc4afbb8dda87287a7eac0 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:36604)
I20260807 01:47:02.877040 27350 tablet_copy_service.cc:161] P fcf4e45defb94276bcfa68392711dade: Beginning new tablet copy session on tablet 2c3fdef784cc4afbb8dda87287a7eac0 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:36604: session id = 57bb4985157447f0a3ab680fa0026f12-2c3fdef784cc4afbb8dda87287a7eac0
I20260807 01:47:02.880071 29172 ts_tablet_manager.cc:933] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:02.882032 29172 tablet_copy_client.cc:323] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.2:34855
I20260807 01:47:02.883610 27250 tablet_copy_service.cc:140] P 3478729505b046c2a0051c2d0130873f: Received BeginTabletCopySession request for tablet 60a03ab231464be89d6cce9e6988cfba from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:41230)
I20260807 01:47:02.884163 27250 tablet_copy_service.cc:161] P 3478729505b046c2a0051c2d0130873f: Beginning new tablet copy session on tablet 60a03ab231464be89d6cce9e6988cfba from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:41230: session id = 57bb4985157447f0a3ab680fa0026f12-60a03ab231464be89d6cce9e6988cfba
I20260807 01:47:02.887393 27350 tablet_copy_source_session.cc:215] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:02.892441 29170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c3fdef784cc4afbb8dda87287a7eac0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:02.894266 27250 tablet_copy_source_session.cc:215] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:02.907544 29172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60a03ab231464be89d6cce9e6988cfba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:02.923477 29170 tablet_copy_client.cc:806] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:02.924623 29170 tablet_copy_client.cc:670] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:02.930949 29170 tablet_copy_client.cc:538] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:02.949862 29170 tablet_bootstrap.cc:492] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:02.979803 29172 tablet_copy_client.cc:806] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:02.980463 29172 tablet_copy_client.cc:670] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:02.985605 29172 tablet_copy_client.cc:538] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:02.987445 29170 tablet_bootstrap.cc:492] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:02.988512 29170 tablet_bootstrap.cc:492] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:02.989384 29170 ts_tablet_manager.cc:1403] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.040s user 0.027s sys 0.011s
I20260807 01:47:02.992739 29170 raft_consensus.cc:359] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:02.994282 29170 raft_consensus.cc:740] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:02.995985 29170 consensus_queue.cc:260] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:02.999565 29172 tablet_bootstrap.cc:492] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:03.011905 29179 ts_tablet_manager.cc:933] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:03.023756 29170 ts_tablet_manager.cc:1434] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.034s user 0.009s sys 0.009s
I20260807 01:47:03.026916 27350 tablet_copy_service.cc:342] P fcf4e45defb94276bcfa68392711dade: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-2c3fdef784cc4afbb8dda87287a7eac0 received from {username='slave'} at 127.0.0.1:36604
I20260807 01:47:03.027375 27350 tablet_copy_service.cc:434] P fcf4e45defb94276bcfa68392711dade: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-2c3fdef784cc4afbb8dda87287a7eac0 on tablet 2c3fdef784cc4afbb8dda87287a7eac0 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:03.038841 29179 tablet_copy_client.cc:323] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.2:34855
I20260807 01:47:03.040777 27250 tablet_copy_service.cc:140] P 3478729505b046c2a0051c2d0130873f: Received BeginTabletCopySession request for tablet bf07cc968f3343869d12c10e81ce75f9 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:41230)
I20260807 01:47:03.041363 27250 tablet_copy_service.cc:161] P 3478729505b046c2a0051c2d0130873f: Beginning new tablet copy session on tablet bf07cc968f3343869d12c10e81ce75f9 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:41230: session id = 57bb4985157447f0a3ab680fa0026f12-bf07cc968f3343869d12c10e81ce75f9
I20260807 01:47:03.043548 29172 tablet_bootstrap.cc:492] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:03.044593 29172 tablet_bootstrap.cc:492] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:03.045358 29172 ts_tablet_manager.cc:1403] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.046s user 0.021s sys 0.004s
I20260807 01:47:03.049157 29172 raft_consensus.cc:359] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:03.050025 27250 tablet_copy_source_session.cc:215] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:03.050047 29172 raft_consensus.cc:740] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:03.051287 29172 consensus_queue.cc:260] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:03.058588 29172 ts_tablet_manager.cc:1434] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.013s user 0.009s sys 0.002s
I20260807 01:47:03.061172 29179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf07cc968f3343869d12c10e81ce75f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:03.061579 27250 tablet_copy_service.cc:342] P 3478729505b046c2a0051c2d0130873f: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-60a03ab231464be89d6cce9e6988cfba received from {username='slave'} at 127.0.0.1:41230
I20260807 01:47:03.062104 27250 tablet_copy_service.cc:434] P 3478729505b046c2a0051c2d0130873f: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-60a03ab231464be89d6cce9e6988cfba on tablet 60a03ab231464be89d6cce9e6988cfba with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:03.084884 29179 tablet_copy_client.cc:806] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:03.085600 29179 tablet_copy_client.cc:670] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:03.086552 29172 ts_tablet_manager.cc:933] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:03.090891 29179 tablet_copy_client.cc:538] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:03.097898 29172 tablet_copy_client.cc:323] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.2:34855
I20260807 01:47:03.101118 27250 tablet_copy_service.cc:140] P 3478729505b046c2a0051c2d0130873f: Received BeginTabletCopySession request for tablet 498dfe25861a4b4fbf2aca930b357274 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:41230)
I20260807 01:47:03.101709 27250 tablet_copy_service.cc:161] P 3478729505b046c2a0051c2d0130873f: Beginning new tablet copy session on tablet 498dfe25861a4b4fbf2aca930b357274 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:41230: session id = 57bb4985157447f0a3ab680fa0026f12-498dfe25861a4b4fbf2aca930b357274
I20260807 01:47:03.105259 29179 tablet_bootstrap.cc:492] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:03.115861 27250 tablet_copy_source_session.cc:215] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:03.119081 29172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 498dfe25861a4b4fbf2aca930b357274. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:03.141390 29179 tablet_bootstrap.cc:492] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:03.143523 29179 tablet_bootstrap.cc:492] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:03.144570 29179 ts_tablet_manager.cc:1403] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.040s user 0.021s sys 0.008s
I20260807 01:47:03.147845 29179 raft_consensus.cc:359] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:03.148718 29179 raft_consensus.cc:740] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:03.149495 29179 consensus_queue.cc:260] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:03.155447 29179 ts_tablet_manager.cc:1434] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.011s user 0.005s sys 0.004s
I20260807 01:47:03.158905 27250 tablet_copy_service.cc:342] P 3478729505b046c2a0051c2d0130873f: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-bf07cc968f3343869d12c10e81ce75f9 received from {username='slave'} at 127.0.0.1:41230
I20260807 01:47:03.159574 27250 tablet_copy_service.cc:434] P 3478729505b046c2a0051c2d0130873f: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-bf07cc968f3343869d12c10e81ce75f9 on tablet bf07cc968f3343869d12c10e81ce75f9 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:03.164633 29172 tablet_copy_client.cc:806] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:03.165410 29172 tablet_copy_client.cc:670] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:03.185304 29172 tablet_copy_client.cc:538] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:03.198834 29172 tablet_bootstrap.cc:492] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:03.234612 29172 tablet_bootstrap.cc:492] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:03.236931 29172 tablet_bootstrap.cc:492] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:03.238253 29172 ts_tablet_manager.cc:1403] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.040s user 0.022s sys 0.005s
I20260807 01:47:03.241982 29172 raft_consensus.cc:359] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:03.242898 29172 raft_consensus.cc:740] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:03.243651 29172 consensus_queue.cc:260] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:03.257682 29172 ts_tablet_manager.cc:1434] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.019s user 0.003s sys 0.004s
I20260807 01:47:03.259637 27250 tablet_copy_service.cc:342] P 3478729505b046c2a0051c2d0130873f: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-498dfe25861a4b4fbf2aca930b357274 received from {username='slave'} at 127.0.0.1:41230
I20260807 01:47:03.260149 27250 tablet_copy_service.cc:434] P 3478729505b046c2a0051c2d0130873f: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-498dfe25861a4b4fbf2aca930b357274 on tablet 498dfe25861a4b4fbf2aca930b357274 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:03.267532 28715 raft_consensus.cc:1217] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2] Dedup: 2.2->[]
I20260807 01:47:03.327445 28715 raft_consensus.cc:1217] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20260807 01:47:03.440748 28714 raft_consensus.cc:1217] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
W20260807 01:47:03.588812 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 53641.8us
I20260807 01:47:03.615067 28715 raft_consensus.cc:1217] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20260807 01:47:03.804296 29150 raft_consensus.cc:1064] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:03.807245 29150 consensus_queue.cc:237] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:03.816471 29135 raft_consensus.cc:1064] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:03.820123 28715 raft_consensus.cc:1275] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 01:47:03.821903 27102 raft_consensus.cc:1275] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 01:47:03.821974 27333 raft_consensus.cc:1275] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 01:47:03.823788 29135 consensus_queue.cc:237] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:03.822331 29113 consensus_queue.cc:1048] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260807 01:47:03.825559 29159 consensus_queue.cc:1048] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:03.829218 29187 consensus_queue.cc:1048] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:03.844125 28714 raft_consensus.cc:1275] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 01:47:03.846144 29135 consensus_queue.cc:1048] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:03.848574 27230 raft_consensus.cc:1275] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 01:47:03.851222 27102 raft_consensus.cc:1275] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 01:47:03.854972 29101 consensus_queue.cc:1048] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:03.857277 29101 consensus_queue.cc:1048] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:03.857128 29113 raft_consensus.cc:2955] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:03.872522 28714 raft_consensus.cc:2955] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
W20260807 01:47:03.894819 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 68061.3us
W20260807 01:47:03.942353 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 89192.4us
I20260807 01:47:03.942987 26949 catalog_manager.cc:5697] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:03.950754 27102 raft_consensus.cc:2955] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.103994 29101 raft_consensus.cc:2955] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.108656 29150 raft_consensus.cc:1064] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:04.112634 29150 consensus_queue.cc:237] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:04.152885 28711 raft_consensus.cc:2955] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.161631 27230 raft_consensus.cc:2955] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.172401 29159 raft_consensus.cc:1064] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:04.177160 29159 consensus_queue.cc:237] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:04.177913 26950 catalog_manager.cc:5697] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:04.185580 28711 raft_consensus.cc:1275] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:47:04.189580 29193 consensus_queue.cc:1048] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260807 01:47:04.204480 27102 raft_consensus.cc:2955] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.239434 28711 raft_consensus.cc:1275] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 01:47:04.266399 29150 consensus_queue.cc:1048] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260807 01:47:04.279070 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 103756us
I20260807 01:47:04.289273 27333 raft_consensus.cc:1275] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260807 01:47:04.291388 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 97962us
I20260807 01:47:04.291538 27333 raft_consensus.cc:1275] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 01:47:04.291646 27328 raft_consensus.cc:2955] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.297930 29193 consensus_queue.cc:1048] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:04.310647 27330 consensus_queue.cc:237] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:04.313154 27109 raft_consensus.cc:1275] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:47:04.314184 27103 raft_consensus.cc:1275] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 01:47:04.315573 27230 consensus_queue.cc:237] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:04.328212 28713 raft_consensus.cc:1275] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:04.329244 28711 raft_consensus.cc:1275] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 01:47:04.330724 29113 consensus_queue.cc:1048] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:47:04.331391 27103 raft_consensus.cc:1275] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:04.341718 29187 consensus_queue.cc:1048] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:04.344347 29187 consensus_queue.cc:1048] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:47:04.342952 29113 consensus_queue.cc:1048] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260807 01:47:04.352523 28711 raft_consensus.cc:2955] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.353608 29198 consensus_queue.cc:1048] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:04.352947 29113 raft_consensus.cc:2955] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.357725 29219 consensus_queue.cc:1048] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:04.363544 29101 raft_consensus.cc:2955] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.367549 28713 raft_consensus.cc:2955] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.400297 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 60a03ab231464be89d6cce9e6988cfba with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:04.412539 27105 raft_consensus.cc:1275] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 01:47:04.440989 26931 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2c3fdef784cc4afbb8dda87287a7eac0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:04.442826 29135 consensus_queue.cc:1048] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:04.458928 29193 raft_consensus.cc:2955] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.461591 27104 raft_consensus.cc:2955] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.464468 27105 raft_consensus.cc:2955] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.461593 26948 catalog_manager.cc:5697] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 4 to 5, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.491003 27103 raft_consensus.cc:2955] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.516661 27317 tablet_service.cc:1558] Processing DeleteTablet for tablet 60a03ab231464be89d6cce9e6988cfba with delete_type TABLET_DATA_TOMBSTONED (TS fcf4e45defb94276bcfa68392711dade not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:59474
I20260807 01:47:04.555451 26948 catalog_manager.cc:5697] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index 3 to 4, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New cstate: current_term: 2 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:04.587461 27338 raft_consensus.cc:2955] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.593356 28711 raft_consensus.cc:2955] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.599114 29159 raft_consensus.cc:2955] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.645725 27199 tablet_service.cc:1558] Processing DeleteTablet for tablet 2c3fdef784cc4afbb8dda87287a7eac0 with delete_type TABLET_DATA_TOMBSTONED (TS 3478729505b046c2a0051c2d0130873f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53780
I20260807 01:47:04.651332 29235 tablet_replica.cc:333] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade: stopping tablet replica
I20260807 01:47:04.664088 27109 raft_consensus.cc:2955] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.693153 26949 catalog_manager.cc:5697] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:04.737841 28711 raft_consensus.cc:2955] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.755319 27230 consensus_queue.cc:237] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:04.760413 26949 catalog_manager.cc:5697] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.765228 28715 raft_consensus.cc:1275] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:47:04.769871 27333 raft_consensus.cc:1275] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:47:04.769622 27330 raft_consensus.cc:2955] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.791954 29150 consensus_queue.cc:1048] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:04.768535 29236 tablet_replica.cc:333] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f: stopping tablet replica
I20260807 01:47:04.796617 29236 raft_consensus.cc:2243] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:04.798702 29236 raft_consensus.cc:2272] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:04.810257 29216 consensus_queue.cc:1048] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:04.785988 29235 raft_consensus.cc:2243] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:04.812378 29235 raft_consensus.cc:2272] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:04.817768 29235 ts_tablet_manager.cc:1916] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:04.821448 28711 raft_consensus.cc:2955] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.816980 29113 raft_consensus.cc:2955] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.839563 29236 ts_tablet_manager.cc:1916] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:04.838389 27328 raft_consensus.cc:2955] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.843971 27230 consensus_queue.cc:237] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:04.864801 28711 raft_consensus.cc:1275] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:04.861442 26950 catalog_manager.cc:5697] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.866830 29159 consensus_queue.cc:1048] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:47:04.850119 29235 ts_tablet_manager.cc:1929] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260807 01:47:04.876252 29235 log.cc:1199] T 60a03ab231464be89d6cce9e6988cfba P fcf4e45defb94276bcfa68392711dade: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-2-root/wals/60a03ab231464be89d6cce9e6988cfba
I20260807 01:47:04.878635 27328 raft_consensus.cc:1275] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:04.880420 26931 catalog_manager.cc:5028] TS fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): tablet 60a03ab231464be89d6cce9e6988cfba (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:04.884979 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bf07cc968f3343869d12c10e81ce75f9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:04.885183 27088 tablet_service.cc:1558] Processing DeleteTablet for tablet bf07cc968f3343869d12c10e81ce75f9 with delete_type TABLET_DATA_TOMBSTONED (TS 3262a9d183914b699227fdba0ca5bed3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41078
I20260807 01:47:04.889827 29193 raft_consensus.cc:2955] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.890704 29227 consensus_queue.cc:1048] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:04.899741 28715 raft_consensus.cc:2955] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.904382 27330 raft_consensus.cc:2955] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.916460 29238 tablet_replica.cc:333] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3: stopping tablet replica
I20260807 01:47:04.918080 29238 raft_consensus.cc:2243] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:04.918998 29238 raft_consensus.cc:2272] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:04.930240 29238 ts_tablet_manager.cc:1916] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:04.931509 29236 ts_tablet_manager.cc:1929] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260807 01:47:04.932233 29236 log.cc:1199] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3478729505b046c2a0051c2d0130873f: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-1-root/wals/2c3fdef784cc4afbb8dda87287a7eac0
I20260807 01:47:04.934739 26932 catalog_manager.cc:5028] TS 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): tablet 2c3fdef784cc4afbb8dda87287a7eac0 (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:04.936486 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 498dfe25861a4b4fbf2aca930b357274 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:04.938479 26948 catalog_manager.cc:5697] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 4 to 5, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:04.954231 29238 ts_tablet_manager.cc:1929] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 01:47:04.954857 29238 log.cc:1199] T bf07cc968f3343869d12c10e81ce75f9 P 3262a9d183914b699227fdba0ca5bed3: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-0-root/wals/bf07cc968f3343869d12c10e81ce75f9
I20260807 01:47:04.956849 26932 catalog_manager.cc:5028] TS 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): tablet bf07cc968f3343869d12c10e81ce75f9 (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:04.959851 27088 tablet_service.cc:1558] Processing DeleteTablet for tablet 498dfe25861a4b4fbf2aca930b357274 with delete_type TABLET_DATA_TOMBSTONED (TS 3262a9d183914b699227fdba0ca5bed3 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:41078
I20260807 01:47:04.961045 29238 tablet_replica.cc:333] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3: stopping tablet replica
I20260807 01:47:04.982297 29238 raft_consensus.cc:2243] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:04.983271 29238 raft_consensus.cc:2272] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:04.987493 29238 ts_tablet_manager.cc:1916] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:05.002826 29238 ts_tablet_manager.cc:1929] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260807 01:47:05.003162 29238 log.cc:1199] T 498dfe25861a4b4fbf2aca930b357274 P 3262a9d183914b699227fdba0ca5bed3: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-0-root/wals/498dfe25861a4b4fbf2aca930b357274
I20260807 01:47:05.004699 26932 catalog_manager.cc:5028] TS 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): tablet 498dfe25861a4b4fbf2aca930b357274 (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:06.050172 27212 consensus_queue.cc:237] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:06.065258 27330 raft_consensus.cc:1275] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 01:47:06.065840 27109 raft_consensus.cc:1275] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260807 01:47:06.066267 27175 consensus_peers.cc:597] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8b00b70a17a64e288e95507deb091fc1. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:06.067328 29193 consensus_queue.cc:1048] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:47:06.068261 29216 consensus_queue.cc:1048] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:47:06.077638 29193 raft_consensus.cc:2955] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:06.079303 27109 raft_consensus.cc:2955] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:06.081952 27330 raft_consensus.cc:2955] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:06.095368 27109 consensus_queue.cc:237] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:06.100162 26950 catalog_manager.cc:5697] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:06.107972 27330 raft_consensus.cc:1275] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260807 01:47:06.109508 27067 consensus_peers.cc:597] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b18526f467154c49b830de9b1b03066f. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:06.112571 29074 consensus_queue.cc:1048] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:06.116508 27226 raft_consensus.cc:1275] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:06.141482 29074 raft_consensus.cc:2955] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Committing config change with OpId 4.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:06.141506 29252 consensus_queue.cc:1048] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260807 01:47:06.150908 27330 raft_consensus.cc:2955] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Committing config change with OpId 4.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:06.158058 27226 raft_consensus.cc:2955] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Committing config change with OpId 4.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:06.176503 27104 consensus_queue.cc:237] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:06.187862 27328 raft_consensus.cc:1275] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260807 01:47:06.189569 27067 consensus_peers.cc:597] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 965af9378c6a4617b1d50f229ffc2c5b. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:06.189571 29074 consensus_queue.cc:1048] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:47:06.191008 27226 raft_consensus.cc:1275] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 01:47:06.192368 29250 consensus_queue.cc:1048] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:47:06.208572 29074 raft_consensus.cc:2955] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:06.209915 27328 raft_consensus.cc:2955] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:06.220569 26952 catalog_manager.cc:5697] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 4 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:06.238734 27212 consensus_queue.cc:237] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:06.246174 27226 raft_consensus.cc:2955] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:06.250092 26950 catalog_manager.cc:5697] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260807 01:47:06.262612 27175 consensus_peers.cc:597] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 89dc6a602e9e42399d533f67d4bd9bef. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:06.268136 27328 raft_consensus.cc:1275] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:06.269665 27109 raft_consensus.cc:1275] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:06.270865 29216 consensus_queue.cc:1048] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:47:06.279266 29241 consensus_queue.cc:1048] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:47:06.284134 29193 raft_consensus.cc:2955] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:06.286820 27330 raft_consensus.cc:2955] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:06.311224 26948 catalog_manager.cc:5697] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:06.323690 27104 raft_consensus.cc:2955] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:06.601158 29269 ts_tablet_manager.cc:933] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:47:06.609103 29269 tablet_copy_client.cc:323] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.1:35499
I20260807 01:47:06.611474 27127 tablet_copy_service.cc:140] P 3262a9d183914b699227fdba0ca5bed3: Received BeginTabletCopySession request for tablet b18526f467154c49b830de9b1b03066f from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:52556)
I20260807 01:47:06.612141 27127 tablet_copy_service.cc:161] P 3262a9d183914b699227fdba0ca5bed3: Beginning new tablet copy session on tablet b18526f467154c49b830de9b1b03066f from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:52556: session id = 57bb4985157447f0a3ab680fa0026f12-b18526f467154c49b830de9b1b03066f
I20260807 01:47:06.621625 27127 tablet_copy_source_session.cc:215] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:06.625809 29269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b18526f467154c49b830de9b1b03066f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:06.641188 29269 tablet_copy_client.cc:806] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:06.641912 29269 tablet_copy_client.cc:670] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:06.646399 29269 tablet_copy_client.cc:538] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:06.660612 29269 tablet_bootstrap.cc:492] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:06.671427 29271 ts_tablet_manager.cc:933] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:47:06.689870 29271 tablet_copy_client.cc:323] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.1:35499
I20260807 01:47:06.691994 27127 tablet_copy_service.cc:140] P 3262a9d183914b699227fdba0ca5bed3: Received BeginTabletCopySession request for tablet 965af9378c6a4617b1d50f229ffc2c5b from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:52556)
I20260807 01:47:06.692663 27127 tablet_copy_service.cc:161] P 3262a9d183914b699227fdba0ca5bed3: Beginning new tablet copy session on tablet 965af9378c6a4617b1d50f229ffc2c5b from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:52556: session id = 57bb4985157447f0a3ab680fa0026f12-965af9378c6a4617b1d50f229ffc2c5b
I20260807 01:47:06.697949 29276 ts_tablet_manager.cc:933] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:06.715139 27127 tablet_copy_source_session.cc:215] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:06.717785 29276 tablet_copy_client.cc:323] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.2:34855
I20260807 01:47:06.722078 27250 tablet_copy_service.cc:140] P 3478729505b046c2a0051c2d0130873f: Received BeginTabletCopySession request for tablet 8b00b70a17a64e288e95507deb091fc1 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:41230)
I20260807 01:47:06.722694 27250 tablet_copy_service.cc:161] P 3478729505b046c2a0051c2d0130873f: Beginning new tablet copy session on tablet 8b00b70a17a64e288e95507deb091fc1 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:41230: session id = 57bb4985157447f0a3ab680fa0026f12-8b00b70a17a64e288e95507deb091fc1
I20260807 01:47:06.724037 29271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 965af9378c6a4617b1d50f229ffc2c5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:06.731448 27250 tablet_copy_source_session.cc:215] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:06.735227 29276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b00b70a17a64e288e95507deb091fc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:06.745165 29269 tablet_bootstrap.cc:492] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:06.751519 29269 tablet_bootstrap.cc:492] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:06.751613 29271 tablet_copy_client.cc:806] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:06.752486 29271 tablet_copy_client.cc:670] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:06.752816 29269 ts_tablet_manager.cc:1403] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.093s user 0.016s sys 0.015s
I20260807 01:47:06.756666 29271 tablet_copy_client.cc:538] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:06.757367 29269 raft_consensus.cc:359] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:06.758368 29269 raft_consensus.cc:740] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:06.759109 29269 consensus_queue.cc:260] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:06.759989 29276 tablet_copy_client.cc:806] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:06.760627 29276 tablet_copy_client.cc:670] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:06.770299 29271 tablet_bootstrap.cc:492] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:06.773985 29276 tablet_copy_client.cc:538] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:06.783723 29276 tablet_bootstrap.cc:492] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:06.797830 29269 ts_tablet_manager.cc:1434] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.045s user 0.008s sys 0.000s
I20260807 01:47:06.818957 27127 tablet_copy_service.cc:342] P 3262a9d183914b699227fdba0ca5bed3: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-b18526f467154c49b830de9b1b03066f received from {username='slave'} at 127.0.0.1:52556
I20260807 01:47:06.819468 27127 tablet_copy_service.cc:434] P 3262a9d183914b699227fdba0ca5bed3: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-b18526f467154c49b830de9b1b03066f on tablet b18526f467154c49b830de9b1b03066f with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:06.831142 29276 tablet_bootstrap.cc:492] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:06.832334 29276 tablet_bootstrap.cc:492] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:06.833218 29276 ts_tablet_manager.cc:1403] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.050s user 0.026s sys 0.003s
I20260807 01:47:06.833710 29271 tablet_bootstrap.cc:492] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:06.845646 29271 tablet_bootstrap.cc:492] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:06.847043 29271 ts_tablet_manager.cc:1403] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.077s user 0.017s sys 0.021s
I20260807 01:47:06.847501 29276 raft_consensus.cc:359] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:06.848479 29276 raft_consensus.cc:740] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:06.849184 29276 consensus_queue.cc:260] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:06.852384 29276 ts_tablet_manager.cc:1434] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.019s user 0.005s sys 0.001s
I20260807 01:47:06.852010 29271 raft_consensus.cc:359] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:06.853027 29271 raft_consensus.cc:740] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:06.853832 29271 consensus_queue.cc:260] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:06.854808 27250 tablet_copy_service.cc:342] P 3478729505b046c2a0051c2d0130873f: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-8b00b70a17a64e288e95507deb091fc1 received from {username='slave'} at 127.0.0.1:41230
I20260807 01:47:06.855305 27250 tablet_copy_service.cc:434] P 3478729505b046c2a0051c2d0130873f: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-8b00b70a17a64e288e95507deb091fc1 on tablet 8b00b70a17a64e288e95507deb091fc1 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:06.856580 29271 ts_tablet_manager.cc:1434] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.009s user 0.004s sys 0.004s
I20260807 01:47:06.857326 29269 ts_tablet_manager.cc:933] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:06.862532 27127 tablet_copy_service.cc:342] P 3262a9d183914b699227fdba0ca5bed3: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-965af9378c6a4617b1d50f229ffc2c5b received from {username='slave'} at 127.0.0.1:52556
I20260807 01:47:06.863049 27127 tablet_copy_service.cc:434] P 3262a9d183914b699227fdba0ca5bed3: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-965af9378c6a4617b1d50f229ffc2c5b on tablet 965af9378c6a4617b1d50f229ffc2c5b with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:06.863605 29269 tablet_copy_client.cc:323] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.2:34855
I20260807 01:47:06.865957 27250 tablet_copy_service.cc:140] P 3478729505b046c2a0051c2d0130873f: Received BeginTabletCopySession request for tablet 89dc6a602e9e42399d533f67d4bd9bef from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:41230)
I20260807 01:47:06.866557 27250 tablet_copy_service.cc:161] P 3478729505b046c2a0051c2d0130873f: Beginning new tablet copy session on tablet 89dc6a602e9e42399d533f67d4bd9bef from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:41230: session id = 57bb4985157447f0a3ab680fa0026f12-89dc6a602e9e42399d533f67d4bd9bef
I20260807 01:47:06.878043 27250 tablet_copy_source_session.cc:215] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:06.886933 29269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89dc6a602e9e42399d533f67d4bd9bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:06.907430 29269 tablet_copy_client.cc:806] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:06.908131 29269 tablet_copy_client.cc:670] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:06.912346 29269 tablet_copy_client.cc:538] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:06.929473 29269 tablet_bootstrap.cc:492] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:06.955888 29269 tablet_bootstrap.cc:492] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:06.956931 29269 tablet_bootstrap.cc:492] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:06.961105 29269 ts_tablet_manager.cc:1403] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.029s user 0.027s sys 0.001s
I20260807 01:47:06.964962 29269 raft_consensus.cc:359] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:06.966018 29269 raft_consensus.cc:740] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:06.966845 29269 consensus_queue.cc:260] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:06.969655 29269 ts_tablet_manager.cc:1434] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.007s user 0.002s sys 0.003s
I20260807 01:47:06.975091 27250 tablet_copy_service.cc:342] P 3478729505b046c2a0051c2d0130873f: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-89dc6a602e9e42399d533f67d4bd9bef received from {username='slave'} at 127.0.0.1:41230
I20260807 01:47:06.975601 27250 tablet_copy_service.cc:434] P 3478729505b046c2a0051c2d0130873f: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-89dc6a602e9e42399d533f67d4bd9bef on tablet 89dc6a602e9e42399d533f67d4bd9bef with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:07.056922 28715 raft_consensus.cc:1217] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.2->[4.3-4.3] Dedup: 4.3->[]
I20260807 01:47:07.073365 28715 raft_consensus.cc:1217] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260807 01:47:07.187359 28715 raft_consensus.cc:1217] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260807 01:47:07.230589 28715 raft_consensus.cc:1217] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2] Dedup: 2.2->[]
I20260807 01:47:07.678272 29208 raft_consensus.cc:1064] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:07.680758 29208 consensus_queue.cc:237] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:07.702736 29289 raft_consensus.cc:1064] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:07.713069 29289 consensus_queue.cc:237] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:07.728631 29252 raft_consensus.cc:1064] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:07.731114 29252 consensus_queue.cc:237] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 1, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:07.739535 28715 raft_consensus.cc:1275] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:47:07.739698 28714 raft_consensus.cc:1275] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:47:07.741348 29193 consensus_queue.cc:1048] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:07.741348 29292 consensus_queue.cc:1048] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:07.742364 27327 raft_consensus.cc:1275] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:47:07.744654 29193 consensus_queue.cc:1048] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260807 01:47:07.768659 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 63699.9us
I20260807 01:47:07.771736 27330 raft_consensus.cc:1275] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:47:07.778625 29122 consensus_queue.cc:1048] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260807 01:47:07.782581 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 75630us
I20260807 01:47:07.790959 27212 raft_consensus.cc:1275] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:47:07.803640 27101 raft_consensus.cc:1275] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:47:07.805438 28711 raft_consensus.cc:1275] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:47:07.808238 29122 consensus_queue.cc:1048] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:07.822625 29122 consensus_queue.cc:1048] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:07.826179 27212 raft_consensus.cc:1275] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260807 01:47:07.830808 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 49948.2us
I20260807 01:47:07.833314 27338 raft_consensus.cc:1275] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:47:07.834546 29216 consensus_queue.cc:1048] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:07.840443 29193 raft_consensus.cc:1064] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:07.843233 29193 consensus_queue.cc:237] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:07.844847 29294 consensus_queue.cc:1048] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:07.849689 29303 consensus_queue.cc:1048] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:07.866248 29289 raft_consensus.cc:2955] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:07.871915 29301 raft_consensus.cc:2955] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:07.872673 27101 raft_consensus.cc:2955] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:07.878190 28711 raft_consensus.cc:1275] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 01:47:07.880268 29122 raft_consensus.cc:2955] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Committing config change with OpId 4.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:07.881026 29216 consensus_queue.cc:1048] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:07.895454 27330 raft_consensus.cc:1275] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 01:47:07.894594 27338 raft_consensus.cc:2955] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:07.898679 27212 raft_consensus.cc:2955] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:07.906749 28714 raft_consensus.cc:2955] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:07.908699 26948 catalog_manager.cc:5697] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:07.906787 27230 raft_consensus.cc:2955] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:07.930958 29216 consensus_queue.cc:1048] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:07.935078 27337 raft_consensus.cc:2955] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:07.900077 28715 raft_consensus.cc:2955] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:07.954859 27326 raft_consensus.cc:2955] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:07.970058 26952 catalog_manager.cc:5697] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:07.970106 28713 raft_consensus.cc:2955] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
W20260807 01:47:08.105654 29331 log.cc:927] Time spent T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.089s user 0.001s sys 0.000s
I20260807 01:47:08.115535 26949 catalog_manager.cc:5697] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
W20260807 01:47:08.123296 29330 log.cc:927] Time spent T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df: Append to log took a long time: real 0.055s user 0.001s sys 0.001s
I20260807 01:47:08.231026 27101 raft_consensus.cc:1275] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 01:47:08.269165 29193 raft_consensus.cc:2955] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:08.306265 28715 raft_consensus.cc:2955] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:08.342648 27104 consensus_queue.cc:237] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:08.346138 27064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:08.352480 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:08.357301 26950 catalog_manager.cc:5697] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:08.379089 27104 consensus_queue.cc:237] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:08.381793 28715 raft_consensus.cc:1275] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 01:47:08.383589 29252 consensus_queue.cc:1048] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:08.399065 29122 raft_consensus.cc:2955] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 4 to 5, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:08.412457 28711 raft_consensus.cc:2955] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:08.419760 29323 consensus_queue.cc:1048] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260807 01:47:08.420962 27330 raft_consensus.cc:1275] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 01:47:08.426268 27330 raft_consensus.cc:2955] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:08.447672 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b18526f467154c49b830de9b1b03066f with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:08.451511 29074 consensus_queue.cc:1048] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:47:08.465627 28715 raft_consensus.cc:1275] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:47:08.467252 29252 consensus_queue.cc:1048] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:08.482381 27328 raft_consensus.cc:2955] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:08.497471 27330 raft_consensus.cc:1275] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:47:08.503427 27212 consensus_queue.cc:237] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:08.512208 28711 raft_consensus.cc:1275] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:47:08.516114 29335 consensus_queue.cc:1048] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260807 01:47:08.520778 29074 consensus_queue.cc:1048] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:08.526935 26952 catalog_manager.cc:5697] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index 4 to 5, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New cstate: current_term: 4 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:08.518872 29292 raft_consensus.cc:2955] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:08.539489 27328 raft_consensus.cc:2955] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:08.538554 28711 raft_consensus.cc:2955] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:08.555822 27212 consensus_queue.cc:237] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:08.557258 27101 raft_consensus.cc:1275] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:47:08.573345 27104 raft_consensus.cc:2955] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
W20260807 01:47:08.578729 27265 debug-util.cc:398] Leaking SignalData structure 0x7b0800185400 after lost signal to thread 24600
W20260807 01:47:08.580476 27265 debug-util.cc:398] Leaking SignalData structure 0x7b0800184200 after lost signal to thread 27011
I20260807 01:47:08.581223 29241 raft_consensus.cc:2955] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
W20260807 01:47:08.589805 27265 debug-util.cc:398] Leaking SignalData structure 0x7b080013e6c0 after lost signal to thread 27146
W20260807 01:47:08.594478 27265 debug-util.cc:398] Leaking SignalData structure 0x7b0800333b60 after lost signal to thread 27269
W20260807 01:47:08.595980 27265 debug-util.cc:398] Leaking SignalData structure 0x7b08001ba980 after lost signal to thread 27378
I20260807 01:47:08.596547 27337 raft_consensus.cc:1275] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
W20260807 01:47:08.597661 27265 debug-util.cc:398] Leaking SignalData structure 0x7b0800327720 after lost signal to thread 28748
I20260807 01:47:08.598590 29323 consensus_queue.cc:1048] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:08.608909 28714 raft_consensus.cc:2955] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:08.614951 28715 raft_consensus.cc:1275] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 01:47:08.617671 26952 catalog_manager.cc:5697] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index 3 to 4, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:08.624617 29335 consensus_queue.cc:1048] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:08.625560 29289 consensus_queue.cc:1048] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:08.634974 27199 tablet_service.cc:1558] Processing DeleteTablet for tablet b18526f467154c49b830de9b1b03066f with delete_type TABLET_DATA_TOMBSTONED (TS 3478729505b046c2a0051c2d0130873f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:53780
I20260807 01:47:08.635847 29216 raft_consensus.cc:2955] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:08.640705 27102 raft_consensus.cc:2955] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:08.667666 29340 tablet_replica.cc:333] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f: stopping tablet replica
I20260807 01:47:08.677608 28715 raft_consensus.cc:2955] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:08.685024 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 965af9378c6a4617b1d50f229ffc2c5b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:08.682803 27328 raft_consensus.cc:2955] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:08.709787 27199 tablet_service.cc:1558] Processing DeleteTablet for tablet 965af9378c6a4617b1d50f229ffc2c5b with delete_type TABLET_DATA_TOMBSTONED (TS 3478729505b046c2a0051c2d0130873f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53780
I20260807 01:47:08.715484 29340 raft_consensus.cc:2243] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:08.716389 29340 raft_consensus.cc:2272] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:08.719650 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8b00b70a17a64e288e95507deb091fc1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:08.720454 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 89dc6a602e9e42399d533f67d4bd9bef with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:08.731807 26948 catalog_manager.cc:5697] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:08.750720 26952 catalog_manager.cc:5697] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 3 to 4, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:08.757804 29340 ts_tablet_manager.cc:1916] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:08.785077 27088 tablet_service.cc:1558] Processing DeleteTablet for tablet 89dc6a602e9e42399d533f67d4bd9bef with delete_type TABLET_DATA_TOMBSTONED (TS 3262a9d183914b699227fdba0ca5bed3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41078
I20260807 01:47:08.817623 29344 tablet_replica.cc:333] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3: stopping tablet replica
I20260807 01:47:08.819182 29344 raft_consensus.cc:2243] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:08.820180 29344 raft_consensus.cc:2272] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:08.832653 29344 ts_tablet_manager.cc:1916] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:08.834964 27317 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b00b70a17a64e288e95507deb091fc1 with delete_type TABLET_DATA_TOMBSTONED (TS fcf4e45defb94276bcfa68392711dade not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:59474
I20260807 01:47:08.891563 29346 tablet_replica.cc:333] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade: stopping tablet replica
I20260807 01:47:08.893035 29346 raft_consensus.cc:2243] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:08.894052 29346 raft_consensus.cc:2272] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:08.898303 29346 ts_tablet_manager.cc:1916] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:08.906445 27229 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:08.908185 27229 raft_consensus.cc:606] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:08.910739 29340 ts_tablet_manager.cc:1929] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.4
I20260807 01:47:08.911604 29340 log.cc:1199] T b18526f467154c49b830de9b1b03066f P 3478729505b046c2a0051c2d0130873f: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-1-root/wals/b18526f467154c49b830de9b1b03066f
I20260807 01:47:08.914166 29340 tablet_replica.cc:333] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f: stopping tablet replica
I20260807 01:47:08.915830 29340 raft_consensus.cc:2243] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:08.920037 29340 raft_consensus.cc:2272] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:08.914472 26932 catalog_manager.cc:5028] TS 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): tablet b18526f467154c49b830de9b1b03066f (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:08.925407 29344 ts_tablet_manager.cc:1929] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260807 01:47:08.926198 29344 log.cc:1199] T 89dc6a602e9e42399d533f67d4bd9bef P 3262a9d183914b699227fdba0ca5bed3: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-0-root/wals/89dc6a602e9e42399d533f67d4bd9bef
I20260807 01:47:08.928570 26932 catalog_manager.cc:5028] TS 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): tablet 89dc6a602e9e42399d533f67d4bd9bef (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:08.933466 29340 ts_tablet_manager.cc:1916] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:08.956312 29346 ts_tablet_manager.cc:1929] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 01:47:08.957046 29346 log.cc:1199] T 8b00b70a17a64e288e95507deb091fc1 P fcf4e45defb94276bcfa68392711dade: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-2-root/wals/8b00b70a17a64e288e95507deb091fc1
I20260807 01:47:08.961071 26931 catalog_manager.cc:5028] TS fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): tablet 8b00b70a17a64e288e95507deb091fc1 (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:08.971341 29340 ts_tablet_manager.cc:1929] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 01:47:08.972118 29340 log.cc:1199] T 965af9378c6a4617b1d50f229ffc2c5b P 3478729505b046c2a0051c2d0130873f: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-1-root/wals/965af9378c6a4617b1d50f229ffc2c5b
I20260807 01:47:08.974100 26932 catalog_manager.cc:5028] TS 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): tablet 965af9378c6a4617b1d50f229ffc2c5b (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:08.977754 27102 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:08.978838 27102 raft_consensus.cc:606] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:08.983273 27103 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:08.983922 27103 raft_consensus.cc:606] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:08.987798 27102 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81439f7f101341049bd086b353ca3495"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:08.988440 27102 raft_consensus.cc:606] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:08.991740 27102 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:08.992445 27102 raft_consensus.cc:606] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Received request to transfer leadership to TS 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:09.001251 27102 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:09.001953 27102 raft_consensus.cc:606] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:09.003423 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260807 01:47:09.006390 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:47:09.007633 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:09.039373 29291 raft_consensus.cc:993] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f: : Instructing follower fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:09.040007 29291 raft_consensus.cc:1081] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Signalling peer fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:09.044004 27328 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:59490
I20260807 01:47:09.044636 27328 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:09.044931 27328 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:09.061980 27328 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:09.063743 27328 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:09.066201 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:09.066932 27212 raft_consensus.cc:3055] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:09.067255 27212 raft_consensus.cc:740] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:09.065047 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:09.068071 27212 consensus_queue.cc:260] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:09.068557 27102 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:09.069471 27212 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:09.076368 27102 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:47:09.076838 27212 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:47:09.078130 27287 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:09.080477 29198 raft_consensus.cc:2804] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:47:09.081216 29198 raft_consensus.cc:697] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:09.082191 29198 consensus_queue.cc:237] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:09.109452 26948 catalog_manager.cc:5697] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 1 to 2, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 2 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:47:09.260347 29122 raft_consensus.cc:993] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:09.260919 29252 raft_consensus.cc:1081] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Signalling peer fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:09.269629 27328 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:33976
I20260807 01:47:09.270329 27328 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:09.270686 27328 raft_consensus.cc:3060] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:09.272285 29252 raft_consensus.cc:993] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:09.272905 29122 raft_consensus.cc:1081] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Signalling peer fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:09.276261 27330 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:33976
I20260807 01:47:09.277856 27330 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:09.278285 27330 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:09.278247 27328 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:09.280277 27328 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:09.281111 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:09.281692 27102 raft_consensus.cc:3055] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:09.281991 27102 raft_consensus.cc:740] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:09.282035 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:09.282907 27229 raft_consensus.cc:3060] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:09.282902 27102 consensus_queue.cc:260] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:09.284804 27102 raft_consensus.cc:3060] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:09.287946 27330 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:09.290283 27330 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:09.291531 27102 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:47:09.293578 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:09.293730 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:09.294220 27103 raft_consensus.cc:3055] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:09.294368 27212 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:09.294847 27103 raft_consensus.cc:740] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:09.295820 27103 consensus_queue.cc:260] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:09.296954 27229 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:47:09.294361 27287 leader_election.cc:304] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:09.298610 29198 raft_consensus.cc:2804] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:09.299062 29198 raft_consensus.cc:697] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:09.297223 27103 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:09.300081 29198 consensus_queue.cc:237] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:09.307466 27103 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:47:09.307076 27212 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:47:09.308908 27287 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:09.311333 29333 raft_consensus.cc:2804] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:09.312500 29333 raft_consensus.cc:697] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:09.313427 29333 consensus_queue.cc:237] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:09.319180 26948 catalog_manager.cc:5697] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 3 to 4, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 4 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:47:09.338791 26948 catalog_manager.cc:5697] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 3 to 4, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 4 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:47:09.382885 29122 raft_consensus.cc:993] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:09.383376 29252 raft_consensus.cc:1081] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Signalling peer fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:09.384888 27330 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "81439f7f101341049bd086b353ca3495"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:33976
I20260807 01:47:09.385411 27330 raft_consensus.cc:493] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:09.385708 27330 raft_consensus.cc:3060] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:09.393013 27330 raft_consensus.cc:515] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:09.395061 27330 leader_election.cc:290] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:09.398301 29122 raft_consensus.cc:993] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:09.398075 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:09.398874 27102 raft_consensus.cc:3055] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:09.398864 29122 raft_consensus.cc:1081] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Signalling peer fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:09.399324 27102 raft_consensus.cc:740] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:09.400166 27102 consensus_queue.cc:260] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:09.401312 27102 raft_consensus.cc:3060] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:09.411761 27102 raft_consensus.cc:2468] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:47:09.426745 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:09.427654 27212 raft_consensus.cc:3060] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:09.432420 27287 leader_election.cc:304] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:09.435165 29198 raft_consensus.cc:2804] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:47:09.439939 27338 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:33976
I20260807 01:47:09.440647 27338 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:09.441039 27338 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:09.452790 27338 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:09.454665 27338 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:09.463228 27212 raft_consensus.cc:2468] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:47:09.465997 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:09.466702 27229 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:09.473176 29198 raft_consensus.cc:697] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:09.474339 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:09.474969 27102 raft_consensus.cc:3055] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:09.475260 27102 raft_consensus.cc:740] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:09.480756 27102 consensus_queue.cc:260] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:09.483358 27229 raft_consensus.cc:2468] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:47:09.495746 27287 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:09.493595 29198 consensus_queue.cc:237] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:09.503202 29333 raft_consensus.cc:2804] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:09.504647 29333 raft_consensus.cc:697] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:09.506443 27102 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:09.505849 29333 consensus_queue.cc:237] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:09.508266 29292 raft_consensus.cc:993] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:47:09.508776 29292 raft_consensus.cc:1081] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Signalling peer 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:47:09.514292 28714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:52322
I20260807 01:47:09.514926 28714 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:09.515286 28714 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:09.517263 27102 raft_consensus.cc:2468] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
W20260807 01:47:09.522531 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 44154.3us
I20260807 01:47:09.537691 26948 catalog_manager.cc:5697] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 1 to 2, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 2 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:47:09.551636 28714 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:09.568658 27229 raft_consensus.cc:1275] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:09.575842 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:09.581341 27229 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:09.576506 28714 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:09.589083 29198 consensus_queue.cc:1048] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:47:09.602016 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:09.602712 27103 raft_consensus.cc:3055] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:09.602849 27104 raft_consensus.cc:1275] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:09.603009 27103 raft_consensus.cc:740] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:09.604089 27103 consensus_queue.cc:260] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:09.605367 27103 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:09.606712 26948 catalog_manager.cc:5697] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 3 to 4, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 4 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:47:09.608331 29333 consensus_queue.cc:1048] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:47:09.614488 27103 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 2.
I20260807 01:47:09.620072 28660 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:09.631657 27229 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 2.
I20260807 01:47:09.646803 29374 raft_consensus.cc:2804] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:47:09.677416 29374 raft_consensus.cc:697] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:09.688076 29374 consensus_queue.cc:237] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:09.719342 26948 catalog_manager.cc:5697] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 1 to 2, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4). New cstate: current_term: 2 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:09.763427 27212 raft_consensus.cc:1275] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:09.765254 29198 consensus_queue.cc:1048] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:09.774418 27104 raft_consensus.cc:1275] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:09.775797 29358 consensus_queue.cc:1048] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:09.867027 27212 raft_consensus.cc:1275] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:09.870350 29333 consensus_queue.cc:1048] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:09.929093 27104 raft_consensus.cc:1275] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:09.935940 29333 consensus_queue.cc:1048] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:09.947513 27212 raft_consensus.cc:1275] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:09.955518 29333 consensus_queue.cc:1048] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:47:09.996436 27212 consensus_queue.cc:237] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:09.998770 27102 raft_consensus.cc:1275] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:10.012033 29353 consensus_queue.cc:1048] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:47:10.039657 27229 raft_consensus.cc:1275] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:10.051712 27104 raft_consensus.cc:1275] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 01:47:10.052078 29353 consensus_queue.cc:1048] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:10.053591 29361 consensus_queue.cc:1048] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:10.055160 27326 raft_consensus.cc:1275] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 01:47:10.058826 29360 consensus_queue.cc:1048] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260807 01:47:10.061981 27175 consensus_peers.cc:597] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cf4d8c8825e149adbd4565e0c8fdd109. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:10.063540 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:10.086447 27104 raft_consensus.cc:1275] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:10.101616 29374 consensus_queue.cc:1048] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:47:10.105424 29335 raft_consensus.cc:2955] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:10.108598 27103 raft_consensus.cc:2955] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:10.116863 27229 raft_consensus.cc:1275] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:10.120445 29374 consensus_queue.cc:1048] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:10.184580 27104 raft_consensus.cc:1275] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:10.187563 26948 catalog_manager.cc:5697] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:10.192107 27212 consensus_queue.cc:237] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:10.219308 29198 consensus_queue.cc:1048] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:10.240842 27102 raft_consensus.cc:1275] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 01:47:10.242334 29323 consensus_queue.cc:1048] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260807 01:47:10.253465 27337 raft_consensus.cc:1275] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 01:47:10.253114 27330 raft_consensus.cc:2955] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
W20260807 01:47:10.267623 27175 consensus_peers.cc:597] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b510849c3131475d8c1e032378dc893b. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:10.266533 29361 raft_consensus.cc:2955] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:10.276451 29323 consensus_queue.cc:1048] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:47:10.283265 27103 raft_consensus.cc:2955] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:10.316202 27326 raft_consensus.cc:2955] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:10.318538 26952 catalog_manager.cc:5697] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:10.351034 27330 consensus_queue.cc:237] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:10.408623 27104 raft_consensus.cc:1275] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260807 01:47:10.418046 27289 consensus_peers.cc:597] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1688e72d717b430e86793ecd17b2e952. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:10.419708 27330 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:57092
I20260807 01:47:10.426838 27330 raft_consensus.cc:606] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:10.431047 27229 raft_consensus.cc:1275] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:47:10.433866 27229 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:10.434590 27229 raft_consensus.cc:606] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:10.439005 29198 consensus_queue.cc:1048] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:10.441666 27229 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30561911ad3945159907e917cc1bafa7"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:10.441392 29358 consensus_queue.cc:1048] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:10.442528 27229 raft_consensus.cc:606] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:10.445652 27104 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4098be3cca39416abf4b724092db65e4"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:10.446384 27104 raft_consensus.cc:606] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:10.450016 27104 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b18526f467154c49b830de9b1b03066f"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:10.451323 27104 raft_consensus.cc:606] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Received request to transfer leadership to TS 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:10.466564 27212 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:10.467286 27212 raft_consensus.cc:606] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:10.469877 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260807 01:47:10.471632 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:47:10.472401 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:10.492620 29353 raft_consensus.cc:2955] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:10.496486 27212 raft_consensus.cc:2955] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:10.506533 29413 ts_tablet_manager.cc:933] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:10.507969 27103 raft_consensus.cc:2955] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:10.526674 27104 consensus_queue.cc:237] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:10.549667 27330 raft_consensus.cc:1275] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:47:10.549124 26950 catalog_manager.cc:5697] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 3 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:10.551833 29208 consensus_queue.cc:1048] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260807 01:47:10.554113 27067 consensus_peers.cc:597] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b3ee061f38f84a71ad3991835a2e56dc. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:10.564451 27212 raft_consensus.cc:1275] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:47:10.568660 29413 tablet_copy_client.cc:323] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.2:34855
I20260807 01:47:10.570884 29208 consensus_queue.cc:1048] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:10.573607 27250 tablet_copy_service.cc:140] P 3478729505b046c2a0051c2d0130873f: Received BeginTabletCopySession request for tablet cf4d8c8825e149adbd4565e0c8fdd109 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:41230)
I20260807 01:47:10.574270 27250 tablet_copy_service.cc:161] P 3478729505b046c2a0051c2d0130873f: Beginning new tablet copy session on tablet cf4d8c8825e149adbd4565e0c8fdd109 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:41230: session id = 57bb4985157447f0a3ab680fa0026f12-cf4d8c8825e149adbd4565e0c8fdd109
I20260807 01:47:10.610357 29361 raft_consensus.cc:993] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:10.611017 29335 raft_consensus.cc:1081] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:10.617774 27104 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "30561911ad3945159907e917cc1bafa7"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:41080
I20260807 01:47:10.618470 27104 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:10.618860 27104 raft_consensus.cc:3060] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:10.619040 27250 tablet_copy_source_session.cc:215] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:10.627616 29252 raft_consensus.cc:2955] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:10.629134 27229 raft_consensus.cc:2955] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:10.641286 27104 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:10.645423 27104 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:10.650004 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:10.651245 27230 raft_consensus.cc:3055] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:10.651932 27230 raft_consensus.cc:740] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:10.653236 27230 consensus_queue.cc:260] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:10.655313 27230 raft_consensus.cc:3060] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:10.671006 29413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf4d8c8825e149adbd4565e0c8fdd109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:10.672617 26952 catalog_manager.cc:5697] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:10.693140 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:10.695048 27337 raft_consensus.cc:3060] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:10.703449 29363 raft_consensus.cc:993] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f: : Instructing follower fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:10.704968 29363 raft_consensus.cc:1081] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Signalling peer fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:10.705906 29363 raft_consensus.cc:993] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:10.710848 27337 raft_consensus.cc:2468] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:10.733637 27230 raft_consensus.cc:2468] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:10.731817 29360 raft_consensus.cc:1081] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:10.746833 27104 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:41080
I20260807 01:47:10.747637 27104 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:10.748000 27104 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:10.742488 27063 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:10.767577 29368 raft_consensus.cc:2804] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:47:10.768131 29368 raft_consensus.cc:697] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:10.769073 29368 consensus_queue.cc:237] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:10.779215 27104 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:10.783150 29358 raft_consensus.cc:993] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:10.783797 29358 raft_consensus.cc:1081] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:10.787513 27103 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:34468
I20260807 01:47:10.788199 27103 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:10.788565 27103 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:10.800750 27103 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:10.804103 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:10.804764 27230 raft_consensus.cc:3055] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:10.805045 27230 raft_consensus.cc:740] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:10.805608 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:10.806196 27337 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:10.806223 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:10.806988 27212 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:10.807542 27103 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:10.807384 27104 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:10.808497 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:10.809052 27328 raft_consensus.cc:3055] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:10.808741 27230 consensus_queue.cc:260] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:10.809379 27328 raft_consensus.cc:740] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
W20260807 01:47:10.817252 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 58677.2us
I20260807 01:47:10.818688 27326 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:59490
I20260807 01:47:10.819303 27326 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:10.819648 27326 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:10.822830 27230 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:10.823606 27330 raft_consensus.cc:2955] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:10.819900 27328 consensus_queue.cc:260] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:10.858541 27328 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:10.863358 27337 raft_consensus.cc:2468] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:10.866191 29426 ts_tablet_manager.cc:933] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:10.869693 27212 raft_consensus.cc:2468] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:47:10.896636 27230 raft_consensus.cc:2468] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:10.896656 27326 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:10.900185 27328 raft_consensus.cc:2468] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:47:10.907390 27064 leader_election.cc:304] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:10.909099 27063 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:10.909684 26948 catalog_manager.cc:5697] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 2, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:10.914060 29208 raft_consensus.cc:2804] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:10.914605 29208 raft_consensus.cc:697] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:10.915437 29208 consensus_queue.cc:237] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:10.919703 27326 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:10.920673 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:10.921329 27104 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:10.913141 29292 raft_consensus.cc:2804] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:47:10.931363 29292 raft_consensus.cc:697] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:10.932420 29292 consensus_queue.cc:237] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:10.947567 27104 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:47:10.952119 27287 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:10.957072 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:10.957820 27212 raft_consensus.cc:3055] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:10.958639 29353 raft_consensus.cc:2804] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:47:10.959115 29423 ts_tablet_manager.cc:933] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:10.959230 29353 raft_consensus.cc:697] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:10.962440 29420 raft_consensus.cc:993] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:10.963001 29420 raft_consensus.cc:1081] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Signalling peer fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:10.958117 27212 raft_consensus.cc:740] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:10.968252 27212 consensus_queue.cc:260] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:10.969241 27328 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4098be3cca39416abf4b724092db65e4"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:33976
I20260807 01:47:10.969828 27328 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:10.970237 27328 raft_consensus.cc:3060] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:10.975505 29413 tablet_copy_client.cc:806] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:10.976123 27212 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:10.976109 29413 tablet_copy_client.cc:670] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:10.975317 29353 consensus_queue.cc:237] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:10.978274 27328 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:11.002096 29426 tablet_copy_client.cc:323] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.3:42429
I20260807 01:47:11.004310 27350 tablet_copy_service.cc:140] P fcf4e45defb94276bcfa68392711dade: Received BeginTabletCopySession request for tablet 1688e72d717b430e86793ecd17b2e952 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:36604)
I20260807 01:47:11.004914 27350 tablet_copy_service.cc:161] P fcf4e45defb94276bcfa68392711dade: Beginning new tablet copy session on tablet 1688e72d717b430e86793ecd17b2e952 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:36604: session id = 57bb4985157447f0a3ab680fa0026f12-1688e72d717b430e86793ecd17b2e952
I20260807 01:47:11.006322 29423 tablet_copy_client.cc:323] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.2:34855
I20260807 01:47:10.997906 27212 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:47:11.015626 26948 catalog_manager.cc:5697] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 3 to 4, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 4 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:11.017359 26948 catalog_manager.cc:5697] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 2, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:11.031098 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:11.031790 27104 raft_consensus.cc:3055] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:11.032079 27104 raft_consensus.cc:740] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:11.033066 27104 consensus_queue.cc:260] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:11.034435 27104 raft_consensus.cc:3060] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:11.036110 27350 tablet_copy_source_session.cc:215] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:11.038534 29208 raft_consensus.cc:993] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:47:11.039703 29208 raft_consensus.cc:1081] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Signalling peer 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:47:11.038136 27328 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:11.046478 28711 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b18526f467154c49b830de9b1b03066f"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:52322
I20260807 01:47:11.047094 28711 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:11.047437 28711 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:11.051890 27246 tablet_copy_service.cc:140] P 3478729505b046c2a0051c2d0130873f: Received BeginTabletCopySession request for tablet b510849c3131475d8c1e032378dc893b from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:41230)
I20260807 01:47:11.052557 27246 tablet_copy_service.cc:161] P 3478729505b046c2a0051c2d0130873f: Beginning new tablet copy session on tablet b510849c3131475d8c1e032378dc893b from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:41230: session id = 57bb4985157447f0a3ab680fa0026f12-b510849c3131475d8c1e032378dc893b
I20260807 01:47:11.066649 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:11.067368 27226 raft_consensus.cc:3060] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:11.077827 29413 tablet_copy_client.cc:538] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:11.081451 28711 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:11.100076 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:11.100440 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:11.100723 27102 raft_consensus.cc:3055] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:11.090951 29426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1688e72d717b430e86793ecd17b2e952. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:11.094695 27226 raft_consensus.cc:2468] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:47:11.101118 27337 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:11.101101 27102 raft_consensus.cc:740] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:11.103525 29413 tablet_bootstrap.cc:492] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:11.105651 27287 leader_election.cc:304] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:11.103520 27102 consensus_queue.cc:260] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:11.107280 27102 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:11.108201 27246 tablet_copy_source_session.cc:215] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:11.112246 27337 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
I20260807 01:47:11.115995 28655 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:11.117022 29374 raft_consensus.cc:2804] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:47:11.120810 29374 raft_consensus.cc:697] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:11.121801 29374 consensus_queue.cc:237] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:11.130513 27102 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
I20260807 01:47:11.137463 26952 catalog_manager.cc:5697] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 4 to 5, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4). New cstate: current_term: 5 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:11.139354 26948 catalog_manager.cc:5697] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 1 to 2, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 2 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:47:11.142079 28711 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:11.142601 29423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b510849c3131475d8c1e032378dc893b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:11.143415 29436 ts_tablet_manager.cc:933] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:47:11.112434 27104 raft_consensus.cc:2468] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:47:11.113041 29358 raft_consensus.cc:2804] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:47:11.160713 29358 raft_consensus.cc:697] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:11.161921 29358 consensus_queue.cc:237] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:11.173380 29413 tablet_bootstrap.cc:492] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:11.178225 29413 tablet_bootstrap.cc:492] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:11.180016 29413 ts_tablet_manager.cc:1403] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.077s user 0.023s sys 0.013s
I20260807 01:47:11.183904 29426 tablet_copy_client.cc:806] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:11.184634 29426 tablet_copy_client.cc:670] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:11.188953 29426 tablet_copy_client.cc:538] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:11.200598 26948 catalog_manager.cc:5697] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 1 to 2, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 2 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:47:11.202766 29426 tablet_bootstrap.cc:492] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:11.206068 29436 tablet_copy_client.cc:323] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.1:35499
I20260807 01:47:11.208236 27127 tablet_copy_service.cc:140] P 3262a9d183914b699227fdba0ca5bed3: Received BeginTabletCopySession request for tablet b3ee061f38f84a71ad3991835a2e56dc from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:52556)
I20260807 01:47:11.208829 27127 tablet_copy_service.cc:161] P 3262a9d183914b699227fdba0ca5bed3: Beginning new tablet copy session on tablet b3ee061f38f84a71ad3991835a2e56dc from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:52556: session id = 57bb4985157447f0a3ab680fa0026f12-b3ee061f38f84a71ad3991835a2e56dc
I20260807 01:47:11.210055 29413 raft_consensus.cc:359] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:11.213272 29413 raft_consensus.cc:740] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:11.214102 29413 consensus_queue.cc:260] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:11.222723 27226 raft_consensus.cc:1275] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:11.224334 29368 consensus_queue.cc:1048] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:47:11.226969 27127 tablet_copy_source_session.cc:215] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:11.226986 29413 ts_tablet_manager.cc:1434] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.046s user 0.008s sys 0.000s
I20260807 01:47:11.232695 27246 tablet_copy_service.cc:342] P 3478729505b046c2a0051c2d0130873f: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-cf4d8c8825e149adbd4565e0c8fdd109 received from {username='slave'} at 127.0.0.1:41230
I20260807 01:47:11.233312 27246 tablet_copy_service.cc:434] P 3478729505b046c2a0051c2d0130873f: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-cf4d8c8825e149adbd4565e0c8fdd109 on tablet cf4d8c8825e149adbd4565e0c8fdd109 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:11.240518 29436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3ee061f38f84a71ad3991835a2e56dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:11.250972 29423 tablet_copy_client.cc:806] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:11.251647 29423 tablet_copy_client.cc:670] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:11.262388 29423 tablet_copy_client.cc:538] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:11.266789 27337 raft_consensus.cc:1275] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:11.274729 29368 consensus_queue.cc:1048] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:47:11.286723 29423 tablet_bootstrap.cc:492] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:11.373123 29436 tablet_copy_client.cc:806] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:11.373190 29426 tablet_bootstrap.cc:492] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:11.373720 27337 raft_consensus.cc:1275] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:11.373980 29436 tablet_copy_client.cc:670] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:11.375202 29368 consensus_queue.cc:1048] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:47:11.388157 29436 tablet_copy_client.cc:538] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:11.398411 27102 raft_consensus.cc:1275] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:11.401340 29436 tablet_bootstrap.cc:492] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:11.374514 29426 tablet_bootstrap.cc:492] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:11.406032 27328 raft_consensus.cc:1275] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:11.398300 29423 tablet_bootstrap.cc:492] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:11.406066 29358 consensus_queue.cc:1048] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:47:11.408291 29423 tablet_bootstrap.cc:492] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:11.409196 29423 ts_tablet_manager.cc:1403] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.123s user 0.024s sys 0.013s
I20260807 01:47:11.412634 29423 raft_consensus.cc:359] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:11.413774 29423 raft_consensus.cc:740] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:11.414463 29423 consensus_queue.cc:260] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:11.413803 29368 consensus_queue.cc:1048] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260807 01:47:11.425943 29423 ts_tablet_manager.cc:1434] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.016s user 0.007s sys 0.000s
I20260807 01:47:11.431344 29426 ts_tablet_manager.cc:1403] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.230s user 0.027s sys 0.011s
I20260807 01:47:11.435228 27246 tablet_copy_service.cc:342] P 3478729505b046c2a0051c2d0130873f: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-b510849c3131475d8c1e032378dc893b received from {username='slave'} at 127.0.0.1:41230
I20260807 01:47:11.436769 27246 tablet_copy_service.cc:434] P 3478729505b046c2a0051c2d0130873f: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-b510849c3131475d8c1e032378dc893b on tablet b510849c3131475d8c1e032378dc893b with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:11.443750 29426 raft_consensus.cc:359] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:11.451920 29426 raft_consensus.cc:740] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:11.460696 28711 raft_consensus.cc:1217] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20260807 01:47:11.463776 29426 consensus_queue.cc:260] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:11.466635 29426 ts_tablet_manager.cc:1434] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.029s user 0.003s sys 0.004s
I20260807 01:47:11.472438 27350 tablet_copy_service.cc:342] P fcf4e45defb94276bcfa68392711dade: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-1688e72d717b430e86793ecd17b2e952 received from {username='slave'} at 127.0.0.1:36604
I20260807 01:47:11.472898 27350 tablet_copy_service.cc:434] P fcf4e45defb94276bcfa68392711dade: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-1688e72d717b430e86793ecd17b2e952 on tablet 1688e72d717b430e86793ecd17b2e952 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:11.490267 29436 tablet_bootstrap.cc:492] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:11.491492 29436 tablet_bootstrap.cc:492] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:11.492357 29436 ts_tablet_manager.cc:1403] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.091s user 0.027s sys 0.010s
I20260807 01:47:11.495594 29436 raft_consensus.cc:359] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:11.496735 29436 raft_consensus.cc:740] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:11.497464 29436 consensus_queue.cc:260] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:11.503335 29436 ts_tablet_manager.cc:1434] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.011s user 0.000s sys 0.006s
W20260807 01:47:11.603627 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 70446.3us
W20260807 01:47:11.616751 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 101474us
I20260807 01:47:11.624701 27226 raft_consensus.cc:1275] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:11.630540 27109 raft_consensus.cc:1275] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 01:47:11.517318 27127 tablet_copy_service.cc:342] P 3262a9d183914b699227fdba0ca5bed3: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-b3ee061f38f84a71ad3991835a2e56dc received from {username='slave'} at 127.0.0.1:52556
I20260807 01:47:11.663177 27127 tablet_copy_service.cc:434] P 3262a9d183914b699227fdba0ca5bed3: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-b3ee061f38f84a71ad3991835a2e56dc on tablet b3ee061f38f84a71ad3991835a2e56dc with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:11.633854 27230 raft_consensus.cc:1275] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:11.642302 27328 raft_consensus.cc:1275] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 01:47:11.664718 29374 consensus_queue.cc:1048] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 01:47:11.666229 29374 consensus_queue.cc:1048] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 01:47:11.688166 27230 raft_consensus.cc:1275] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:11.692521 29292 consensus_queue.cc:1048] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260807 01:47:11.706651 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 46149.5us
I20260807 01:47:11.710271 29421 consensus_queue.cc:1048] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:11.752938 27226 raft_consensus.cc:1275] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:11.780754 29333 consensus_queue.cc:1048] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:47:11.782331 29358 consensus_queue.cc:1048] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:47:11.814590 27104 raft_consensus.cc:1275] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:11.837598 29358 consensus_queue.cc:1048] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:47:11.871157 29335 raft_consensus.cc:1064] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:11.874202 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:11.873842 29335 consensus_queue.cc:237] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:11.888789 28714 raft_consensus.cc:1217] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
W20260807 01:47:12.081308 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 75054.1us
I20260807 01:47:12.100134 28714 raft_consensus.cc:1217] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3] Dedup: 3.3->[]
I20260807 01:47:12.118747 28711 raft_consensus.cc:1217] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3] Dedup: 3.3->[]
I20260807 01:47:12.137773 28714 raft_consensus.cc:1275] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 01:47:12.165092 29425 consensus_queue.cc:1048] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:12.222616 27104 raft_consensus.cc:1275] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 01:47:12.246887 27338 raft_consensus.cc:1275] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 01:47:12.255748 27338 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4098be3cca39416abf4b724092db65e4"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:57092
I20260807 01:47:12.256417 27338 raft_consensus.cc:606] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:12.266110 27229 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:12.267037 27229 raft_consensus.cc:606] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Received request to transfer leadership to TS 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:12.270313 27229 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:12.271070 27229 raft_consensus.cc:606] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:12.274864 27102 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:12.275463 27102 raft_consensus.cc:606] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:12.278753 27102 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:12.279317 27102 raft_consensus.cc:606] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Received request to transfer leadership to TS 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:12.282315 27102 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:12.283007 27102 raft_consensus.cc:606] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Received request to transfer leadership to TS 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:12.284482 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260807 01:47:12.285851 29419 raft_consensus.cc:993] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:12.286405 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:47:12.286614 29419 raft_consensus.cc:1081] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:12.295161 29335 consensus_queue.cc:1048] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:12.306469 27102 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:41080
I20260807 01:47:12.307065 27102 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:12.307451 27102 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:12.287276 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:12.316959 27102 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:12.318923 27102 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:12.335475 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:12.336208 27229 raft_consensus.cc:3055] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:12.336736 27229 raft_consensus.cc:740] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:12.337980 27229 consensus_queue.cc:260] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:12.345889 27229 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:12.358525 29425 raft_consensus.cc:1064] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:12.352819 29363 raft_consensus.cc:2955] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:12.369694 29425 consensus_queue.cc:237] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:12.372805 29485 consensus_queue.cc:1048] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:12.404464 28711 raft_consensus.cc:2955] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:12.408565 29358 raft_consensus.cc:993] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:12.409400 29358 raft_consensus.cc:1081] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:12.425900 26952 catalog_manager.cc:5697] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:12.432447 27105 raft_consensus.cc:2955] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:12.435405 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:12.436028 27330 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:12.439355 27338 raft_consensus.cc:2955] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:12.450733 28710 raft_consensus.cc:1275] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:47:12.453332 29363 consensus_queue.cc:1048] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:12.462725 27330 raft_consensus.cc:2468] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:12.480511 27105 raft_consensus.cc:1275] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:47:12.491143 27229 raft_consensus.cc:2468] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:12.493197 27105 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4098be3cca39416abf4b724092db65e4"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:34468
I20260807 01:47:12.494014 27105 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:12.494470 27105 raft_consensus.cc:3060] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:12.508771 29425 consensus_queue.cc:1048] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:12.518087 27226 consensus_queue.cc:237] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:12.521343 27105 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:12.511807 27064 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:12.524215 29421 raft_consensus.cc:2804] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:47:12.524966 29421 raft_consensus.cc:697] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:12.526137 29421 consensus_queue.cc:237] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:12.531168 27330 raft_consensus.cc:1275] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:47:12.536530 27105 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:12.541816 28711 raft_consensus.cc:1275] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:12.543715 29292 raft_consensus.cc:1064] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:12.545679 29485 consensus_queue.cc:1048] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:12.543766 29363 consensus_queue.cc:1048] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260807 01:47:12.547878 29292 consensus_queue.cc:237] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:12.553092 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:12.553814 27229 raft_consensus.cc:3060] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:12.554476 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:12.555058 27328 raft_consensus.cc:3055] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:12.555346 27328 raft_consensus.cc:740] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:12.556069 27328 consensus_queue.cc:260] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:12.557253 27328 raft_consensus.cc:3060] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:12.557821 27102 raft_consensus.cc:1275] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:12.567420 28711 raft_consensus.cc:2955] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:12.570102 27328 raft_consensus.cc:2468] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:47:12.563072 27229 raft_consensus.cc:2468] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:47:12.582082 29363 consensus_queue.cc:1048] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:12.584126 27063 leader_election.cc:304] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:12.594250 27102 raft_consensus.cc:2955] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:12.609645 29477 raft_consensus.cc:993] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:47:12.611711 29208 raft_consensus.cc:1081] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Signalling peer 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:47:12.613315 26948 catalog_manager.cc:5697] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 4 to 5, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 5 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:12.630059 29198 raft_consensus.cc:1064] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:12.632375 26952 catalog_manager.cc:5697] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 4 to 5, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:12.614773 29368 raft_consensus.cc:2804] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:47:12.632927 29198 consensus_queue.cc:237] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:12.653400 29368 raft_consensus.cc:697] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:12.670393 28711 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:52322
I20260807 01:47:12.671077 28711 raft_consensus.cc:493] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:12.671450 29368 consensus_queue.cc:237] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:12.565804 29485 raft_consensus.cc:2955] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:12.671450 28711 raft_consensus.cc:3060] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:12.694839 28711 raft_consensus.cc:515] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:12.697312 28711 leader_election.cc:290] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:12.708518 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:12.709227 27109 raft_consensus.cc:3055] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:12.709560 27109 raft_consensus.cc:740] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:12.716944 29420 raft_consensus.cc:993] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:47:12.717826 29420 raft_consensus.cc:1081] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Signalling peer 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:47:12.722508 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:12.723171 27326 raft_consensus.cc:3060] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:12.713050 27109 consensus_queue.cc:260] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:12.733927 27109 raft_consensus.cc:3060] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:12.735965 27326 raft_consensus.cc:2468] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 2.
I20260807 01:47:12.746061 28655 leader_election.cc:304] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:12.747143 29374 raft_consensus.cc:2804] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:47:12.747701 28714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:52322
I20260807 01:47:12.748610 28714 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:12.749246 28714 raft_consensus.cc:3060] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:12.747737 27105 raft_consensus.cc:2955] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:12.753813 27109 raft_consensus.cc:2468] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 2.
I20260807 01:47:12.747781 29374 raft_consensus.cc:697] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:12.770828 29323 raft_consensus.cc:2955] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
W20260807 01:47:12.766024 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 111836us
I20260807 01:47:12.778666 29374 consensus_queue.cc:237] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:12.814972 27317 tablet_service.cc:1558] Processing DeleteTablet for tablet cf4d8c8825e149adbd4565e0c8fdd109 with delete_type TABLET_DATA_TOMBSTONED (TS fcf4e45defb94276bcfa68392711dade not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:59474
I20260807 01:47:12.827282 28714 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:12.843082 27326 raft_consensus.cc:2955] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:12.850834 28714 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:12.894395 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:12.895136 27109 raft_consensus.cc:3055] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:12.895469 27109 raft_consensus.cc:740] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:12.905776 27330 raft_consensus.cc:1275] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 01:47:12.937510 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:12.938364 27326 raft_consensus.cc:3060] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:12.952554 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cf4d8c8825e149adbd4565e0c8fdd109 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:12.957998 28714 raft_consensus.cc:2955] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:12.960348 27229 raft_consensus.cc:1275] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 01:47:12.962632 27326 raft_consensus.cc:2468] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 2.
I20260807 01:47:12.966797 28655 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:12.968963 27109 consensus_queue.cc:260] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:12.970342 29374 raft_consensus.cc:2804] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:47:12.971465 28711 raft_consensus.cc:1275] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 01:47:12.985251 26952 catalog_manager.cc:5697] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
W20260807 01:47:12.998181 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 98244.1us
W20260807 01:47:13.000160 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 102141us
I20260807 01:47:12.977978 26948 catalog_manager.cc:5697] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 2 to 3, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
W20260807 01:47:13.004420 29507 log.cc:927] Time spent T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.085s user 0.000s sys 0.002s
I20260807 01:47:13.022382 29519 tablet_replica.cc:333] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade: stopping tablet replica
I20260807 01:47:13.024430 29519 raft_consensus.cc:2243] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:13.025631 29519 raft_consensus.cc:2272] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:12.999143 29292 consensus_queue.cc:1048] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260807 01:47:13.031555 27109 raft_consensus.cc:3060] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:13.032425 26950 catalog_manager.cc:5697] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 1 to 2, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4). New cstate: current_term: 2 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:13.004524 29358 consensus_queue.cc:1048] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260807 01:47:13.052084 28710 raft_consensus.cc:1275] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 01:47:13.055687 29512 raft_consensus.cc:993] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f: : Instructing follower 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:47:13.063483 27109 raft_consensus.cc:2468] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 2.
I20260807 01:47:13.080789 29519 ts_tablet_manager.cc:1916] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:13.070466 29425 raft_consensus.cc:1081] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Signalling peer 57bb4985157447f0a3ab680fa0026f12 to start an election
W20260807 01:47:13.156302 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 212416us
I20260807 01:47:13.171054 29527 consensus_queue.cc:1048] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:13.193069 27109 raft_consensus.cc:1275] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 01:47:13.158789 29374 raft_consensus.cc:697] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:13.248818 29198 consensus_queue.cc:1048] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:13.262777 29374 consensus_queue.cc:237] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:13.276101 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 263617us
I20260807 01:47:13.278137 28714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:48350
I20260807 01:47:13.278970 28714 raft_consensus.cc:493] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:13.279335 28714 raft_consensus.cc:3060] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:13.328380 27229 raft_consensus.cc:1275] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 01:47:13.331329 28714 raft_consensus.cc:515] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:13.338521 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 344506us
I20260807 01:47:13.346647 27328 raft_consensus.cc:1275] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:13.381102 29466 consensus_queue.cc:1048] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:13.405318 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:13.406129 27229 raft_consensus.cc:3055] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:13.406527 27229 raft_consensus.cc:740] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:13.407284 27229 consensus_queue.cc:260] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:13.408123 27109 raft_consensus.cc:1275] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:13.408396 27229 raft_consensus.cc:3060] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:13.414840 28714 leader_election.cc:290] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:13.424331 27229 raft_consensus.cc:2468] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
W20260807 01:47:13.442262 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 104136us
I20260807 01:47:13.450269 27226 raft_consensus.cc:1275] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 01:47:13.456362 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:13.457165 27337 raft_consensus.cc:3060] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:13.471467 27337 raft_consensus.cc:2468] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:47:13.496630 28655 leader_election.cc:304] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
W20260807 01:47:13.553407 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 64333.6us
I20260807 01:47:13.555796 29533 raft_consensus.cc:2804] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:47:13.556417 29533 raft_consensus.cc:697] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:13.559406 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:13.625298 29548 consensus_queue.cc:1048] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.154s
I20260807 01:47:13.634481 29520 raft_consensus.cc:993] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:13.557364 29533 consensus_queue.cc:237] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:13.763607 29501 raft_consensus.cc:1081] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Signalling peer fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:13.822476 29358 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:13.850212 28711 raft_consensus.cc:1240] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3. Ops: []
W20260807 01:47:13.871027 29552 log.cc:927] Time spent T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.063s user 0.000s sys 0.001s
I20260807 01:47:13.823519 29358 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:13.880278 29358 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:13.901593 29519 ts_tablet_manager.cc:1929] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260807 01:47:13.902199 27337 raft_consensus.cc:1240] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 1. Current term is 2. Ops: []
I20260807 01:47:13.905361 27337 raft_consensus.cc:1275] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:13.911401 29519 log.cc:1199] T cf4d8c8825e149adbd4565e0c8fdd109 P fcf4e45defb94276bcfa68392711dade: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-2-root/wals/cf4d8c8825e149adbd4565e0c8fdd109
I20260807 01:47:13.919409 29530 consensus_queue.cc:1048] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:47:13.922729 29519 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59474 (request call id 71) took 1108 ms. Trace:
I20260807 01:47:13.923143 29519 rpcz_store.cc:276] 0807 01:47:12.814603 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:12.814769 (+ 166us) service_pool.cc:224] Handling call
0807 01:47:12.990017 (+175248us) ts_tablet_manager.cc:1151] Deleting tablet cf4d8c8825e149adbd4565e0c8fdd109
0807 01:47:12.992090 (+ 2073us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0807 01:47:13.771897 (+779807us) tablet_metadata.cc:665] Metadata flushed
0807 01:47:13.901411 (+129514us) tablet_metadata.cc:665] Metadata flushed
0807 01:47:13.922609 (+ 21198us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":150382,"thread_start_us":81946,"threads_started":1}
I20260807 01:47:13.928052 27325 raft_consensus.cc:1275] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:47:13.930003 27325 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:33976
I20260807 01:47:13.949366 27325 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:13.949903 27325 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:13.956542 29483 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:13.957249 29483 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:13.959486 29483 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:47:14.039403 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 120967us
I20260807 01:47:13.961757 29530 consensus_queue.cc:1048] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
W20260807 01:47:14.059099 27064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:14.056651 29208 consensus_queue.cc:1048] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260807 01:47:14.056087 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 90637.8us
I20260807 01:47:14.092720 27229 raft_consensus.cc:1275] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:47:14.096184 27100 raft_consensus.cc:1275] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:13.927134 26931 catalog_manager.cc:5028] TS fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): tablet cf4d8c8825e149adbd4565e0c8fdd109 (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
W20260807 01:47:14.101624 26931 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 174503us
I20260807 01:47:14.102885 27338 raft_consensus.cc:1275] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 01:47:14.102766 29466 consensus_queue.cc:1048] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260807 01:47:14.055660 29292 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
W20260807 01:47:14.125639 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 63384.3us
W20260807 01:47:14.126226 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 64515us
I20260807 01:47:14.129202 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:14.122514 29501 raft_consensus.cc:2955] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:14.135125 29527 consensus_queue.cc:1048] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:14.149650 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:14.151561 29568 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:14.161624 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:14.152237 29568 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:14.167554 29568 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:14.170846 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:14.129338 29292 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:14.179370 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 52967.6us
I20260807 01:47:14.144353 29358 raft_consensus.cc:493] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
W20260807 01:47:14.189894 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 56168.2us
I20260807 01:47:14.204991 29527 consensus_queue.cc:1048] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260807 01:47:14.188222 29358 raft_consensus.cc:515] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:14.202131 27213 raft_consensus.cc:2955] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:14.236785 29358 leader_election.cc:290] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:14.191227 27325 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:14.252292 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 107211us
W20260807 01:47:14.253727 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 64804.2us
I20260807 01:47:14.254173 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:14.255749 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:14.279446 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:14.282502 27104 raft_consensus.cc:2955] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:14.293808 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:14.254114 29581 consensus_queue.cc:1048] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260807 01:47:14.300197 27287 leader_election.cc:304] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:14.305403 27287 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:14.248420 29477 consensus_queue.cc:1048] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260807 01:47:14.319028 29292 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:14.313906 26950 catalog_manager.cc:5697] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 1 to 2, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4). New cstate: current_term: 2 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:14.307807 29572 raft_consensus.cc:2749] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:47:14.206064 29483 raft_consensus.cc:2955] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
W20260807 01:47:14.349912 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 44617.6us
W20260807 01:47:14.385771 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 54826us
I20260807 01:47:14.385485 28714 raft_consensus.cc:2955] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:14.399101 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:47:14.338969 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 93884.5us
I20260807 01:47:14.424713 27067 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:14.482043 29208 raft_consensus.cc:2749] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260807 01:47:14.482273 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 57669.4us
I20260807 01:47:14.485198 27104 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30561911ad3945159907e917cc1bafa7"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:14.485944 27104 raft_consensus.cc:606] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:14.498348 29576 raft_consensus.cc:2749] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:14.511409 27230 raft_consensus.cc:1275] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 01:47:14.494766 27325 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:14.543437 29208 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:14.544050 29208 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:14.573639 29208 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:47:14.607087 29586 log.cc:927] Time spent T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.118s user 0.001s sys 0.000s
I20260807 01:47:14.609006 29572 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:14.609572 29572 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:14.642202 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:14.673895 29466 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:14.674635 29466 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:14.693753 29466 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:14.700368 29572 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:14.691969 27325 raft_consensus.cc:1275] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 01:47:14.757261 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:47:14.768887 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 381460us
I20260807 01:47:14.776352 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:14.800490 27229 consensus_queue.cc:237] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:14.803081 29576 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:14.804315 29597 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:14.804245 29576 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:14.807041 29597 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:14.827906 27212 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
W20260807 01:47:14.844316 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 74026.9us
I20260807 01:47:14.846400 27287 leader_election.cc:304] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
W20260807 01:47:14.851799 29483 consensus_meta.cc:287] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade: Time spent flushing consensus metadata: real 0.520s user 0.014s sys 0.000s
I20260807 01:47:14.787395 29572 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:14.855237 29482 raft_consensus.cc:493] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:14.860814 29482 raft_consensus.cc:515] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:14.868417 29593 raft_consensus.cc:2749] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260807 01:47:14.871209 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 47657us
I20260807 01:47:14.876278 29363 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:14.877151 29363 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:14.860782 29597 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:14.883209 29541 raft_consensus.cc:493] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:14.884160 29541 raft_consensus.cc:515] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:14.883278 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:14.892668 29363 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:14.857765 27212 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:14.894102 27212 raft_consensus.cc:606] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:14.896390 29600 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:14.897248 29600 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:14.900027 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:14.918339 29524 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:14.923614 29506 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:14.807003 29530 consensus_queue.cc:1048] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.100s
I20260807 01:47:14.924477 29506 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:14.933241 29576 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:14.919492 29524 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:14.922768 29482 leader_election.cc:290] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:14.943883 29358 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:14.944876 29358 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:14.956954 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:14.958714 29466 consensus_queue.cc:1048] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260807 01:47:14.985769 27325 raft_consensus.cc:2468] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 2.
I20260807 01:47:14.988037 29527 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:14.988962 29527 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:14.988477 29358 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:15.004904 28655 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:15.018007 29585 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:15.039575 29541 leader_election.cc:290] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:15.039299 27100 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:15.018904 29585 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:15.040321 27100 raft_consensus.cc:606] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Received request to transfer leadership to TS 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:15.042578 29585 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:15.049914 29512 raft_consensus.cc:493] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:15.049934 29587 raft_consensus.cc:2804] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260807 01:47:15.049707 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 58329.9us
I20260807 01:47:15.050807 29512 raft_consensus.cc:515] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:15.055929 29587 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
W20260807 01:47:15.046396 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:14.994014 29572 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:15.061312 29587 raft_consensus.cc:3060] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:15.065184 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:15.067057 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:15.068405 29208 raft_consensus.cc:993] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:47:15.071417 27338 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:57092
I20260807 01:47:15.072124 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:15.072127 28660 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:15.085012 29368 raft_consensus.cc:1081] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Signalling peer 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:47:15.080242 27338 raft_consensus.cc:606] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:15.086582 29506 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:15.087325 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:15.088370 29571 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:15.057557 29600 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:15.105538 27101 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:15.106348 27101 raft_consensus.cc:606] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:15.107069 29608 raft_consensus.cc:2749] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:15.106032 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:15.086380 27100 raft_consensus.cc:3055] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:15.110869 29572 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:15.113086 28715 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:52322
W20260807 01:47:15.113788 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 72229.1us
I20260807 01:47:15.113888 28715 raft_consensus.cc:493] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:15.116322 28715 raft_consensus.cc:3060] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:15.115051 29571 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:15.113235 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 62037us
I20260807 01:47:15.108974 27100 raft_consensus.cc:740] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:15.129810 27101 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:15.130785 27101 raft_consensus.cc:606] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Received request to transfer leadership to TS 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:15.131287 27101 raft_consensus.cc:614] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 01:47:15.145329 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:15.150179 29523 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:15.151070 29523 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:15.150828 27212 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:15.168488 29421 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:15.169365 29421 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:15.168556 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:15.126139 27100 consensus_queue.cc:260] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:15.175482 27100 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:15.175089 29421 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:15.175872 29524 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:15.199551 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:15.199945 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:15.200922 29501 raft_consensus.cc:493] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:15.206382 29425 raft_consensus.cc:493] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
W20260807 01:47:15.214435 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 61752.4us
I20260807 01:47:15.201627 29501 raft_consensus.cc:515] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:15.207149 29425 raft_consensus.cc:515] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:15.226295 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:15.243721 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:15.244227 29363 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:15.246716 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:15.249675 27064 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:15.250746 29573 raft_consensus.cc:2749] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:47:15.244884 29363 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:15.259667 29363 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:15.261293 29501 leader_election.cc:290] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:47:15.262045 27170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:15.264901 29587 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:15.266525 28714 raft_consensus.cc:1240] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:15.269976 27226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57662 (request call id 1643) took 1002 ms. Trace:
I20260807 01:47:15.270457 27226 rpcz_store.cc:276] 0807 01:47:14.266980 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:14.267152 (+ 172us) service_pool.cc:224] Handling call
0807 01:47:14.267458 (+ 306us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:47:14.267791 (+ 333us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:14.267817 (+ 26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:47:14.760514 (+492697us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b2c00110c68. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d53d19c71 00007f7d53d295d2 00007f7d55459de1 00007f7d5541e81a 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22
0807 01:47:14.761470 (+ 956us) log.cc:844] Serialized 26 byte log entry
0807 01:47:15.063330 (+301860us) raft_consensus.cc:1645] Marking committed up to 2
0807 01:47:15.074732 (+ 11402us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:15.074784 (+ 52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:47:15.269493 (+194709us) raft_consensus.cc:1685] finished
0807 01:47:15.269513 (+ 20us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:15.269785 (+ 272us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1425,"raft.queue_time_us":329,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":14119,"spinlock_wait_cycles":7724800,"thread_start_us":301270,"threads_started":1,"wal-append.queue_time_us":411439}
I20260807 01:47:15.239198 29523 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:15.276356 29425 leader_election.cc:290] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:15.277567 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:15.244433 28709 raft_consensus.cc:2955] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:15.269533 29512 leader_election.cc:290] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:15.291312 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:15.293543 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:15.295742 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:15.293970 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:47:15.297104 29624 log.cc:927] Time spent T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.093s user 0.000s sys 0.003s
I20260807 01:47:15.304498 27064 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:15.295295 29587 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:15.312973 29368 raft_consensus.cc:2749] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:15.213505 27218 raft_consensus.cc:2955] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:15.304893 28715 raft_consensus.cc:515] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:15.332404 28714 raft_consensus.cc:1240] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 1. Current term is 2. Ops: []
I20260807 01:47:15.334375 27212 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:47:15.276144 29527 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:15.345808 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:15.239447 29571 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:15.353511 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:15.354692 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:15.361368 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:15.362790 27170 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:15.367980 27333 raft_consensus.cc:3060] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:15.376794 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:15.377624 29420 consensus_queue.cc:1059] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260807 01:47:15.382850 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:15.397514 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:15.399824 28715 leader_election.cc:290] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:15.401026 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:15.401461 27326 raft_consensus.cc:1240] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 1. Current term is 2. Ops: []
I20260807 01:47:15.402125 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:15.405871 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:15.409597 27108 raft_consensus.cc:3055] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:15.410053 27108 raft_consensus.cc:740] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:15.411396 28708 raft_consensus.cc:1275] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:47:15.406767 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:15.411010 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:15.424597 29323 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:15.425289 29323 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:15.421949 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:15.429260 27110 raft_consensus.cc:2468] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:47:15.409858 27325 raft_consensus.cc:2955] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
W20260807 01:47:15.435382 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 77013.1us
I20260807 01:47:15.440541 27064 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:15.448365 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:15.411123 27108 consensus_queue.cc:260] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:15.456775 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:15.454876 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:15.459512 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:15.469642 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:15.470971 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:15.474757 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:15.490468 29520 raft_consensus.cc:2749] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260807 01:47:15.491333 29635 log.cc:927] Time spent T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.130s user 0.002s sys 0.000s
I20260807 01:47:15.498910 29323 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:15.473481 29541 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:15.508958 27108 raft_consensus.cc:3060] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:15.511757 27063 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:15.512800 29421 raft_consensus.cc:2749] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:47:15.513967 29323 raft_consensus.cc:993] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:15.514544 29323 raft_consensus.cc:1081] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:15.476557 29521 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:15.515182 29521 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:15.517328 29521 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:15.502779 29541 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:15.520538 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 64219us
I20260807 01:47:15.539604 27333 raft_consensus.cc:2468] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 2.
I20260807 01:47:15.520247 27172 leader_election.cc:304] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:15.542517 28655 leader_election.cc:304] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:15.535615 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:15.518083 27100 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:47:15.565927 29541 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:15.577481 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:15.579596 27099 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:41080
I20260807 01:47:15.580273 27099 raft_consensus.cc:493] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:15.580650 27099 raft_consensus.cc:3060] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:15.559917 29512 raft_consensus.cc:2804] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:47:15.571547 29608 raft_consensus.cc:2804] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:47:15.603987 29608 raft_consensus.cc:697] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:15.594514 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:15.605016 29608 consensus_queue.cc:237] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:15.605799 26950 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60866 (request call id 125) took 1443 ms. Trace:
I20260807 01:47:15.590313 29538 consensus_queue.cc:1048] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260807 01:47:15.592293 29524 consensus_queue.cc:1059] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260807 01:47:15.584272 29512 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:15.615291 27287 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
W20260807 01:47:15.147964 24592 auto_leader_rebalancer.cc:467] leader transfer for tablet c8799906440c4ef19cfbadc18cf32ef0 (from 3262a9d183914b699227fdba0ca5bed3 to 57bb4985157447f0a3ab680fa0026f12) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 01:47:15.614562 29512 raft_consensus.cc:3060] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:15.619973 27108 raft_consensus.cc:2468] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 2.
W20260807 01:47:15.634788 29573 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260807 01:47:15.638962 29482 raft_consensus.cc:2804] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:47:15.639640 29482 raft_consensus.cc:697] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:15.640604 29482 consensus_queue.cc:237] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:15.651067 29567 raft_consensus.cc:3055] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:15.651527 29567 raft_consensus.cc:740] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:15.652475 29567 consensus_queue.cc:260] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:15.617506 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260807 01:47:15.653853 29567 raft_consensus.cc:3060] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:15.656258 29580 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:15.665805 27287 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:15.668179 27287 leader_election.cc:304] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:15.686178 29483 raft_consensus.cc:2749] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260807 01:47:15.683074 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 97096.2us
W20260807 01:47:15.660916 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 44645.8us
I20260807 01:47:15.668896 29600 raft_consensus.cc:2749] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:15.692759 29425 raft_consensus.cc:2749] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:15.606366 26950 rpcz_store.cc:276] 0807 01:47:14.126862 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:14.134288 (+ 7426us) service_pool.cc:224] Handling call
0807 01:47:15.560677 (+1426389us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 01:47:14.688634 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 01:47:14.689419 (+ 785us) write_op.cc:432] Acquiring schema lock in shared mode
0807 01:47:14.689458 (+ 39us) write_op.cc:435] Acquired schema lock
0807 01:47:14.689475 (+ 17us) tablet.cc:662] Decoding operations
0807 01:47:14.689864 (+ 389us) write_op.cc:620] Acquiring the partition lock for write op
0807 01:47:14.689922 (+ 58us) write_op.cc:641] Partition lock acquired for write op
0807 01:47:14.689950 (+ 28us) tablet.cc:685] Acquiring locks for 1 operations
0807 01:47:14.691179 (+ 1229us) tablet.cc:701] Row locks acquired
0807 01:47:14.691196 (+ 17us) write_op.cc:260] PREPARE: finished
0807 01:47:14.691346 (+ 150us) write_op.cc:270] Start()
0807 01:47:14.691469 (+ 123us) write_op.cc:276] Timestamp: P: 1786067234691301 usec, L: 0
0807 01:47:14.691485 (+ 16us) op_driver.cc:348] REPLICATION: starting
0807 01:47:14.701371 (+ 9886us) log.cc:844] Serialized 527 byte log entry
0807 01:47:15.094280 (+392909us) op_driver.cc:464] REPLICATION: finished
0807 01:47:15.294347 (+200067us) write_op.cc:301] APPLY: starting
0807 01:47:15.294539 (+ 192us) tablet.cc:1367] starting BulkCheckPresence
0807 01:47:15.306406 (+ 11867us) tablet.cc:1370] finished BulkCheckPresence
0807 01:47:15.306579 (+ 173us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 01:47:15.311451 (+ 4872us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 01:47:15.311844 (+ 393us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0807 01:47:15.312013 (+ 169us) write_op.cc:312] APPLY: finished
0807 01:47:15.312416 (+ 403us) log.cc:844] Serialized 23 byte log entry
0807 01:47:15.312655 (+ 239us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:47:15.312797 (+ 142us) write_op.cc:454] Released schema lock
0807 01:47:15.312852 (+ 55us) write_op.cc:341] FINISH: Updating metrics
0807 01:47:15.379535 (+ 66683us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b54003d2b28. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d5b814c21 00007f7d5b81388c 00007f7d58e89ddd 00007f7d58e75d69 00007f7d58e75533 00007f7d58e79741 00007f7d58e796f9 00007f7d58e79689 00007f7d58e79651 00007f7d58e7887d 00000000003aa554
0807 01:47:15.436963 (+ 57428us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:47:15.438268 (+ 1305us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6418,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":122766,"apply.run_cpu_time_us":7224,"apply.run_wall_time_us":143988,"num_ops":1,"prepare.queue_time_us":363097,"prepare.run_cpu_time_us":9036,"prepare.run_wall_time_us":86665,"raft.queue_time_us":82765,"raft.run_cpu_time_us":12979,"raft.run_wall_time_us":194309,"replication_time_us":337120,"spinlock_wait_cycles":36953344,"thread_start_us":415158,"threads_started":4,"wal-append.queue_time_us":73698}]]}
I20260807 01:47:15.696627 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:47:15.753362 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:15.755882 27170 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:15.708374 27099 raft_consensus.cc:515] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:15.784646 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:15.787506 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:15.660691 29580 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:15.776706 29649 raft_consensus.cc:2749] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:15.792649 27099 leader_election.cc:290] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:15.793440 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
W20260807 01:47:15.798296 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 130280us
I20260807 01:47:15.802287 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:15.810695 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:15.814575 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:15.819199 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:15.821043 29512 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:15.826282 29512 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:15.809406 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:15.831017 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:15.863199 29580 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:15.867820 27230 raft_consensus.cc:3055] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:15.868281 27230 raft_consensus.cc:740] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:15.869079 27230 consensus_queue.cc:260] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:15.891886 27230 raft_consensus.cc:3060] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:15.893151 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:15.919298 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:15.946518 27230 raft_consensus.cc:2468] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:15.948899 27064 leader_election.cc:304] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
W20260807 01:47:15.958631 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 183401us
I20260807 01:47:15.958736 29520 raft_consensus.cc:2804] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election won for term 2
W20260807 01:47:15.951557 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 170832us
I20260807 01:47:15.965349 29520 raft_consensus.cc:697] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:15.973007 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:15.979192 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:15.979923 27105 raft_consensus.cc:3060] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:15.982187 29520 consensus_queue.cc:237] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:15.984628 27328 raft_consensus.cc:1275] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:47:16.005573 27105 raft_consensus.cc:2468] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:47:16.005007 27212 raft_consensus.cc:2468] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:47:16.008797 28660 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:47:16.011209 29587 raft_consensus.cc:2804] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:47:16.030046 27172 leader_election.cc:304] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:16.030841 29425 raft_consensus.cc:2804] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:47:16.031399 29425 raft_consensus.cc:697] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:16.032246 29425 consensus_queue.cc:237] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:16.029912 29587 raft_consensus.cc:697] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:16.036015 29587 consensus_queue.cc:237] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:16.038916 27108 raft_consensus.cc:1240] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 1. Current term is 2. Ops: []
I20260807 01:47:16.040853 29521 raft_consensus.cc:2955] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:16.024554 26950 catalog_manager.cc:5697] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:16.049626 28709 raft_consensus.cc:2955] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:16.058528 26950 catalog_manager.cc:5697] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:16.084636 29521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:53780 (request call id 75) took 1344 ms. Trace:
I20260807 01:47:16.086370 27287 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:16.091842 29483 raft_consensus.cc:2749] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:47:16.093250 27287 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:16.101920 29483 raft_consensus.cc:2749] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:16.108850 27287 leader_election.cc:304] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:16.085136 29521 rpcz_store.cc:276] 0807 01:47:14.740473 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:14.749544 (+ 9071us) service_pool.cc:224] Handling call
0807 01:47:14.873372 (+123828us) log.cc:844] Serialized 480 byte log entry
0807 01:47:15.167262 (+293890us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b54000aa854. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d55424b15 00007f7d5541cdda 00007f7d5a4162d9 00007f7d53960db4 00007f7d53960d42 00007f7d53960c71 00007f7d53960bec 00007f7d5395fea2 00007f7d5375807c 00007f7d537574b6
0807 01:47:16.084520 (+917258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76337,"raft.queue_time_us":23184,"raft.run_cpu_time_us":1711,"raft.run_wall_time_us":32238,"spinlock_wait_cycles":29223808,"thread_start_us":202991,"threads_started":1,"wal-append.queue_time_us":256370}
I20260807 01:47:16.111184 29580 raft_consensus.cc:2749] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:47:16.120523 27287 leader_election.cc:304] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:16.123296 29580 raft_consensus.cc:2749] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:47:16.123977 27103 raft_consensus.cc:1275] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:16.128264 29587 consensus_queue.cc:1048] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:47:16.136620 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b510849c3131475d8c1e032378dc893b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:16.140439 27108 raft_consensus.cc:1240] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2. Ops: []
I20260807 01:47:16.141966 27063 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:16.144078 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:16.146967 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:16.154817 27226 raft_consensus.cc:1240] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2. Ops: []
I20260807 01:47:16.157722 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:16.164083 27170 leader_election.cc:304] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:16.168452 29567 raft_consensus.cc:2749] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:16.168876 27063 leader_election.cc:304] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:16.169334 29573 raft_consensus.cc:2749] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:47:16.170250 29421 raft_consensus.cc:2749] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:47:16.173457 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:16.174106 27330 raft_consensus.cc:3060] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:16.165910 27170 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:16.177255 29512 raft_consensus.cc:2749] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:16.177416 29421 raft_consensus.cc:993] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:16.179028 29421 raft_consensus.cc:1081] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Signalling peer fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:16.179565 27170 leader_election.cc:304] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:16.180855 27170 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:16.181277 29512 raft_consensus.cc:2749] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:47:16.181892 27105 raft_consensus.cc:1275] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:16.183939 29521 raft_consensus.cc:2749] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:16.186121 27170 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:16.186916 29425 raft_consensus.cc:2749] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:16.189488 27170 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:16.192286 27170 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:16.192610 29521 raft_consensus.cc:2749] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:47:16.194780 29567 raft_consensus.cc:2749] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:16.194798 27328 raft_consensus.cc:1275] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:16.182372 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:16.200883 27330 raft_consensus.cc:2468] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:16.205838 27287 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:16.207895 27333 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:33976
I20260807 01:47:16.209201 27333 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:16.209743 27333 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:16.207901 29600 raft_consensus.cc:2749] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:16.215255 29482 consensus_queue.cc:1059] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:47:16.216132 29482 consensus_queue.cc:1243] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer 3478729505b046c2a0051c2d0130873f log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260807 01:47:16.216638 29658 raft_consensus.cc:3055] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:16.217027 29658 raft_consensus.cc:740] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:16.217931 29658 consensus_queue.cc:260] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:16.219215 29658 raft_consensus.cc:3060] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:16.204998 29425 consensus_queue.cc:1048] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260807 01:47:16.225219 26952 catalog_manager.cc:5697] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 1 to 2, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:16.227490 26952 catalog_manager.cc:5697] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 2 to 3, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4). New cstate: current_term: 3 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:16.239494 27333 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:16.206743 29587 consensus_queue.cc:1048] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260807 01:47:16.243587 27333 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:16.253921 27108 consensus_queue.cc:237] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:16.281948 29573 raft_consensus.cc:493] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:16.261984 26949 catalog_manager.cc:5697] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 1 to 2, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4). New cstate: current_term: 2 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:16.285357 29573 raft_consensus.cc:515] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:16.289194 29573 leader_election.cc:290] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:16.248593 27287 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
W20260807 01:47:16.300316 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 51205.6us
I20260807 01:47:16.301534 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:16.303602 29600 raft_consensus.cc:2749] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:16.306751 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:16.309818 27067 leader_election.cc:304] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:16.310205 29600 consensus_queue.cc:1048] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:47:16.312495 29573 raft_consensus.cc:2749] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:47:16.310331 27287 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:16.318763 29600 raft_consensus.cc:2749] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:47:16.312976 27330 raft_consensus.cc:2955] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:16.323729 28709 raft_consensus.cc:1275] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 01:47:16.327131 29520 consensus_queue.cc:1048] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:16.338625 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:16.339156 27103 raft_consensus.cc:3055] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:47:16.339394 27103 raft_consensus.cc:740] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:16.340994 27222 raft_consensus.cc:1275] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:16.341431 27103 consensus_queue.cc:260] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:16.343187 27103 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:16.342314 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:16.344595 27214 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:16.351922 26950 catalog_manager.cc:5697] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 2, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:16.347612 26954 catalog_manager.cc:5697] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:16.366752 27103 raft_consensus.cc:2468] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:47:16.369951 27287 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:16.370836 29358 raft_consensus.cc:2804] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:47:16.371414 29358 raft_consensus.cc:697] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:16.372463 29358 consensus_queue.cc:237] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:16.380396 27330 raft_consensus.cc:1240] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6. Ops: []
I20260807 01:47:16.385061 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:16.394317 27222 raft_consensus.cc:1275] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 01:47:16.396023 29573 consensus_queue.cc:1048] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:16.378494 29600 consensus_queue.cc:1048] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260807 01:47:16.415089 27214 raft_consensus.cc:2468] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:47:16.447835 29421 raft_consensus.cc:2955] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:16.449491 28708 raft_consensus.cc:2955] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:16.460186 27088 tablet_service.cc:1558] Processing DeleteTablet for tablet b510849c3131475d8c1e032378dc893b with delete_type TABLET_DATA_TOMBSTONED (TS 3262a9d183914b699227fdba0ca5bed3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41078
I20260807 01:47:16.512058 29683 tablet_replica.cc:333] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3: stopping tablet replica
I20260807 01:47:16.513343 29683 raft_consensus.cc:2243] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:16.514338 29683 raft_consensus.cc:2272] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:16.517716 26948 catalog_manager.cc:5697] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 1 to 2, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 2 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:47:16.526660 29683 ts_tablet_manager.cc:1916] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:16.527019 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b3ee061f38f84a71ad3991835a2e56dc with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:16.540108 26949 catalog_manager.cc:5697] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 2 to 3, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4). New cstate: current_term: 3 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260807 01:47:16.560635 29681 log.cc:927] Time spent T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.064s user 0.000s sys 0.000s
I20260807 01:47:16.565480 27108 raft_consensus.cc:1275] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:16.565176 27325 consensus_queue.cc:237] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:16.583077 29567 consensus_queue.cc:1048] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:47:16.591564 27328 raft_consensus.cc:1275] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260807 01:47:16.610374 29678 log.cc:927] Time spent T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.080s user 0.002s sys 0.000s
I20260807 01:47:16.616564 27227 raft_consensus.cc:1275] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260807 01:47:16.618604 29587 consensus_queue.cc:1048] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 01:47:16.615207 26949 catalog_manager.cc:5697] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 4 to 5, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
W20260807 01:47:16.642373 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 64298.3us
I20260807 01:47:16.666044 28708 raft_consensus.cc:1275] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 01:47:16.667554 27222 raft_consensus.cc:1275] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:16.670665 27325 raft_consensus.cc:3060] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:16.672408 29358 consensus_queue.cc:1048] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:16.679441 27325 raft_consensus.cc:1275] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260807 01:47:16.681928 26954 catalog_manager.cc:5697] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 5 to 6, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 6 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:47:16.683493 27222 raft_consensus.cc:2955] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:16.695122 29587 consensus_queue.cc:1048] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:47:16.711647 29687 consensus_queue.cc:1048] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:47:16.735818 27108 raft_consensus.cc:1275] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 01:47:16.742435 27333 raft_consensus.cc:1275] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:16.738933 29684 consensus_queue.cc:1048] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260807 01:47:16.752468 27317 tablet_service.cc:1558] Processing DeleteTablet for tablet b3ee061f38f84a71ad3991835a2e56dc with delete_type TABLET_DATA_TOMBSTONED (TS fcf4e45defb94276bcfa68392711dade not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:59474
I20260807 01:47:16.766047 29482 consensus_queue.cc:1048] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260807 01:47:16.767431 29567 consensus_queue.cc:1048] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260807 01:47:16.815783 27103 raft_consensus.cc:1275] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260807 01:47:16.831298 29697 tablet_replica.cc:333] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade: stopping tablet replica
I20260807 01:47:16.832664 29697 raft_consensus.cc:2243] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:16.837059 29358 raft_consensus.cc:2955] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:16.839659 28708 raft_consensus.cc:2955] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:16.847548 29683 ts_tablet_manager.cc:1929] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 01:47:16.848271 29683 log.cc:1199] T b510849c3131475d8c1e032378dc893b P 3262a9d183914b699227fdba0ca5bed3: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-0-root/wals/b510849c3131475d8c1e032378dc893b
W20260807 01:47:16.889124 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 65469.6us
I20260807 01:47:16.904395 27218 raft_consensus.cc:1275] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260807 01:47:16.912492 29703 consensus_queue.cc:1048] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
W20260807 01:47:16.923506 29580 raft_consensus.cc:670] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopping
I20260807 01:47:16.947777 26932 catalog_manager.cc:5028] TS 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): tablet b510849c3131475d8c1e032378dc893b (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:16.949324 29697 raft_consensus.cc:2272] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Raft consensus is shut down!
W20260807 01:47:16.960346 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 63614.1us
I20260807 01:47:16.977758 26931 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1688e72d717b430e86793ecd17b2e952 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:16.980952 29697 ts_tablet_manager.cc:1916] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260807 01:47:16.992203 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 79103.5us
W20260807 01:47:17.060444 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 147079us
I20260807 01:47:17.033907 26948 catalog_manager.cc:5697] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 4 to 5, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New cstate: current_term: 3 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:17.081992 29658 consensus_queue.cc:1048] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260807 01:47:17.094892 27105 raft_consensus.cc:2955] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:17.130115 27199 tablet_service.cc:1558] Processing DeleteTablet for tablet 1688e72d717b430e86793ecd17b2e952 with delete_type TABLET_DATA_TOMBSTONED (TS 3478729505b046c2a0051c2d0130873f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:53780
I20260807 01:47:17.137428 29719 tablet_replica.cc:333] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f: stopping tablet replica
I20260807 01:47:17.138746 29719 raft_consensus.cc:2243] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:17.139580 29719 raft_consensus.cc:2272] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:17.147784 29719 ts_tablet_manager.cc:1916] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:17.187007 29697 ts_tablet_manager.cc:1929] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260807 01:47:17.187573 29697 log.cc:1199] T b3ee061f38f84a71ad3991835a2e56dc P fcf4e45defb94276bcfa68392711dade: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-2-root/wals/b3ee061f38f84a71ad3991835a2e56dc
I20260807 01:47:17.190699 26931 catalog_manager.cc:5028] TS fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): tablet b3ee061f38f84a71ad3991835a2e56dc (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:17.192056 29719 ts_tablet_manager.cc:1929] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260807 01:47:17.192646 29719 log.cc:1199] T 1688e72d717b430e86793ecd17b2e952 P 3478729505b046c2a0051c2d0130873f: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-1-root/wals/1688e72d717b430e86793ecd17b2e952
I20260807 01:47:17.202723 26932 catalog_manager.cc:5028] TS 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): tablet 1688e72d717b430e86793ecd17b2e952 (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:17.348819 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:17.402527 27337 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
mode: GRACEFUL
new_leader_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:57092
I20260807 01:47:17.403156 27337 raft_consensus.cc:606] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Received request to transfer leadership to TS 3478729505b046c2a0051c2d0130873f
I20260807 01:47:17.405920 27105 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:17.406548 27105 raft_consensus.cc:606] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Received request to transfer leadership to TS 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:17.409219 27216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:17.409734 27216 raft_consensus.cc:606] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:17.412408 27216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:17.412869 27216 raft_consensus.cc:606] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:17.415293 27216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b510849c3131475d8c1e032378dc893b"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:17.415756 27216 raft_consensus.cc:606] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Received request to transfer leadership to TS 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:17.416981 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260807 01:47:17.419435 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:47:17.420344 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:17.427397 29520 raft_consensus.cc:993] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:47:17.427963 29520 raft_consensus.cc:1081] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Signalling peer 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:47:17.429757 28708 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:52322
I20260807 01:47:17.430552 28708 raft_consensus.cc:493] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:17.430924 28708 raft_consensus.cc:3060] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:17.449378 28708 raft_consensus.cc:515] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:17.452492 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:17.453125 27105 raft_consensus.cc:3055] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:17.453408 27105 raft_consensus.cc:740] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:17.454134 27105 consensus_queue.cc:260] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:17.455549 27105 raft_consensus.cc:3060] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:17.455984 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:17.456627 27337 raft_consensus.cc:3060] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:17.465462 28708 leader_election.cc:290] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:17.475059 27105 raft_consensus.cc:2468] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:47:17.469764 27337 raft_consensus.cc:2468] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:47:17.476312 28660 leader_election.cc:304] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:17.477662 29661 raft_consensus.cc:2804] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:47:17.478235 29661 raft_consensus.cc:697] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:17.479138 29661 consensus_queue.cc:237] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:17.495886 29567 raft_consensus.cc:993] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:17.496417 29567 raft_consensus.cc:1081] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:17.504678 26949 catalog_manager.cc:5697] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 2 to 3, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4). New cstate: current_term: 3 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:17.506444 27105 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:41080
I20260807 01:47:17.507062 27105 raft_consensus.cc:493] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:17.507457 27105 raft_consensus.cc:3060] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:17.535284 27105 raft_consensus.cc:515] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:17.537803 27105 leader_election.cc:290] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:17.537884 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:17.538261 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:17.538618 27218 raft_consensus.cc:3055] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:17.538909 28708 raft_consensus.cc:3060] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:17.538971 27218 raft_consensus.cc:740] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:17.540310 27218 consensus_queue.cc:260] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:17.541657 27218 raft_consensus.cc:3060] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:17.551353 28708 raft_consensus.cc:2468] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:17.552749 27067 leader_election.cc:304] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:17.553519 29573 raft_consensus.cc:2804] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:47:17.555814 29573 raft_consensus.cc:697] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:17.555321 27218 raft_consensus.cc:2468] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:17.557950 29573 consensus_queue.cc:237] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:17.581418 29701 raft_consensus.cc:993] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade: : Instructing follower 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:17.581974 29701 raft_consensus.cc:1081] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Signalling peer 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:17.583840 27218 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:57654
I20260807 01:47:17.584444 27218 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:17.584805 27218 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:17.589936 26949 catalog_manager.cc:5697] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 2, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:17.593883 27218 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:17.602097 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:17.603255 27105 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:17.609418 27218 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:17.609663 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:17.610786 27337 raft_consensus.cc:3055] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:17.611294 27337 raft_consensus.cc:740] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:17.612267 27337 consensus_queue.cc:260] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:17.613946 27337 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:17.625870 27105 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:17.630332 29715 raft_consensus.cc:993] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:17.630950 29715 raft_consensus.cc:1081] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:17.631006 27172 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:17.636524 29710 raft_consensus.cc:2804] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:17.637284 29710 raft_consensus.cc:697] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:17.638428 29710 consensus_queue.cc:237] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:17.659242 27100 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:41080
I20260807 01:47:17.659854 27100 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:17.660243 27100 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:17.660043 27337 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:17.668339 27100 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:17.671415 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:17.672055 28708 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:17.672883 26949 catalog_manager.cc:5697] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 3 to 4, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 4 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:17.673904 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:17.674557 27218 raft_consensus.cc:3055] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:17.674830 27218 raft_consensus.cc:740] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:17.675607 27218 consensus_queue.cc:260] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:17.676667 27218 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:17.680017 28708 raft_consensus.cc:2468] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:47:17.681497 27067 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:17.682478 29573 raft_consensus.cc:2804] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:17.682961 29573 raft_consensus.cc:697] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:17.683868 29573 consensus_queue.cc:237] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:17.686723 27218 raft_consensus.cc:2468] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:47:17.687279 27100 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:17.701313 26949 catalog_manager.cc:5697] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 3 to 4, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 4 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:17.726037 29710 raft_consensus.cc:993] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f: : Instructing follower 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:47:17.726585 29711 raft_consensus.cc:1081] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Signalling peer 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:47:17.728101 28708 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b510849c3131475d8c1e032378dc893b"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:48350
I20260807 01:47:17.728598 28708 raft_consensus.cc:493] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:17.728981 28708 raft_consensus.cc:3060] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:17.735868 28708 raft_consensus.cc:515] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:17.738135 28708 leader_election.cc:290] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:17.738869 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:17.739329 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:17.739531 27218 raft_consensus.cc:3055] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:17.739908 27218 raft_consensus.cc:740] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:17.739915 27337 raft_consensus.cc:3060] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:17.740685 27218 consensus_queue.cc:260] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:17.741916 27218 raft_consensus.cc:3060] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:17.746982 27337 raft_consensus.cc:2468] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 2.
I20260807 01:47:17.748458 28655 leader_election.cc:304] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:17.749249 29661 raft_consensus.cc:2804] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:47:17.750097 29661 raft_consensus.cc:697] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:17.751142 29661 consensus_queue.cc:237] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:17.753499 27218 raft_consensus.cc:2468] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 2.
I20260807 01:47:17.762588 26949 catalog_manager.cc:5697] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 1 to 2, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4). New cstate: current_term: 2 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:17.865797 27328 raft_consensus.cc:1275] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260807 01:47:17.868839 29661 consensus_queue.cc:1048] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 01:47:17.896900 27108 raft_consensus.cc:1275] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260807 01:47:17.898434 29661 consensus_queue.cc:1048] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 01:47:18.035861 27328 raft_consensus.cc:1275] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:47:18.040617 29710 consensus_queue.cc:1048] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:18.067593 28708 raft_consensus.cc:1275] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 01:47:18.072353 29573 consensus_queue.cc:1048] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:47:18.082038 28708 raft_consensus.cc:1275] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:18.085036 27108 raft_consensus.cc:1275] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:47:18.087617 29711 consensus_queue.cc:1048] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:18.112535 29573 consensus_queue.cc:1048] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
W20260807 01:47:18.147737 29744 log.cc:927] Time spent T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.050s user 0.001s sys 0.000s
I20260807 01:47:18.156854 27216 raft_consensus.cc:1275] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 01:47:18.160211 29573 consensus_queue.cc:1048] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:47:18.164409 27216 raft_consensus.cc:1275] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:18.173645 29421 consensus_queue.cc:1048] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:18.177601 27328 raft_consensus.cc:1275] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:18.179720 29661 consensus_queue.cc:1048] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:47:18.294229 27103 consensus_queue.cc:237] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:18.325284 27218 raft_consensus.cc:1275] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:18.326858 27328 raft_consensus.cc:1275] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 01:47:18.327445 29578 consensus_queue.cc:1048] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260807 01:47:18.328595 27067 consensus_peers.cc:597] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 31708844dc4f41feb6a7a2820cfc0e0b. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:18.329994 29750 consensus_queue.cc:1048] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:18.335435 27216 raft_consensus.cc:1275] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 01:47:18.337437 29750 consensus_queue.cc:1048] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:18.360836 29742 raft_consensus.cc:2955] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:18.364185 27328 raft_consensus.cc:2955] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:18.377480 27218 raft_consensus.cc:2955] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:18.389098 26948 catalog_manager.cc:5697] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 2 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:18.407917 27218 consensus_queue.cc:237] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
W20260807 01:47:18.466854 27175 consensus_peers.cc:597] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 565d0a4463ac45e4bad1886cc0095ff7. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:18.467103 27328 raft_consensus.cc:1275] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:47:18.471119 29715 consensus_queue.cc:1048] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:18.475973 27103 raft_consensus.cc:1275] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:47:18.477433 29715 consensus_queue.cc:1048] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:18.532727 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:18.573935 29746 raft_consensus.cc:2955] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:18.580766 27337 raft_consensus.cc:2955] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:18.600212 27331 consensus_queue.cc:237] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:18.605535 27103 raft_consensus.cc:2955] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:18.609983 26950 catalog_manager.cc:5697] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index -1 to 4, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 4 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260807 01:47:18.633818 27289 consensus_peers.cc:597] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 08edf043c99f445b99e723211e8e0ad0. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:18.634491 27218 raft_consensus.cc:1275] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:47:18.636673 27103 raft_consensus.cc:1275] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:47:18.636665 29716 consensus_queue.cc:1048] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:18.639865 29658 consensus_queue.cc:1048] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:18.651417 27100 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:18.652053 27100 raft_consensus.cc:606] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:18.655651 27331 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
mode: GRACEFUL
new_leader_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:57092
I20260807 01:47:18.656365 27331 raft_consensus.cc:606] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Received request to transfer leadership to TS 3478729505b046c2a0051c2d0130873f
I20260807 01:47:18.662356 27218 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:18.662993 27218 raft_consensus.cc:606] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:18.664829 29716 raft_consensus.cc:2955] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:18.666595 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260807 01:47:18.669047 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:47:18.669919 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:18.673372 27216 raft_consensus.cc:2955] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:18.685593 27103 raft_consensus.cc:2955] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
W20260807 01:47:18.709017 27036 auto_rebalancer.cc:644] Failed to schedule move for tablet 40f160d4dff84d8d9fb21b3489d01ff7: Remote error: Service unavailable: leader transfer in progress
I20260807 01:47:18.723532 26948 catalog_manager.cc:5697] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index -1 to 4, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 4 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:18.773881 29778 raft_consensus.cc:993] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade: : Instructing follower 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:18.774500 29778 raft_consensus.cc:1081] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Signalling peer 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:18.776932 27216 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:57654
I20260807 01:47:18.777505 27216 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:18.777869 27216 raft_consensus.cc:3060] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:18.783978 27216 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:18.786861 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:18.787124 27216 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:18.787533 27100 raft_consensus.cc:3060] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:18.789376 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:18.790004 27337 raft_consensus.cc:3055] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:18.790388 27337 raft_consensus.cc:740] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:18.791239 27337 consensus_queue.cc:260] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:18.792366 27337 raft_consensus.cc:3060] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:18.795037 27100 raft_consensus.cc:2468] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:18.796396 27172 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:18.797262 29746 raft_consensus.cc:2804] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:18.797767 29746 raft_consensus.cc:697] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:18.798627 29746 consensus_queue.cc:237] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:18.801120 27337 raft_consensus.cc:2468] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:18.810612 26949 catalog_manager.cc:5697] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 3 to 4, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 4 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:18.859292 29780 ts_tablet_manager.cc:933] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:18.861580 29780 tablet_copy_client.cc:323] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.2:34855
I20260807 01:47:18.863744 27246 tablet_copy_service.cc:140] P 3478729505b046c2a0051c2d0130873f: Received BeginTabletCopySession request for tablet 565d0a4463ac45e4bad1886cc0095ff7 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:41230)
I20260807 01:47:18.864413 27246 tablet_copy_service.cc:161] P 3478729505b046c2a0051c2d0130873f: Beginning new tablet copy session on tablet 565d0a4463ac45e4bad1886cc0095ff7 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:41230: session id = 57bb4985157447f0a3ab680fa0026f12-565d0a4463ac45e4bad1886cc0095ff7
I20260807 01:47:18.873711 27246 tablet_copy_source_session.cc:215] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:18.878939 29780 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 565d0a4463ac45e4bad1886cc0095ff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:18.883975 29421 raft_consensus.cc:993] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:18.887631 29742 raft_consensus.cc:1081] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Signalling peer fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:18.890574 27337 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:33976
I20260807 01:47:18.891332 27337 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:18.895498 27337 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:18.918596 29780 tablet_copy_client.cc:806] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:18.919291 29782 ts_tablet_manager.cc:933] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:47:18.919375 29780 tablet_copy_client.cc:670] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:18.931479 29780 tablet_copy_client.cc:538] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:18.936033 27337 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:18.943693 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:18.944308 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:18.945005 27216 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:18.944396 27100 raft_consensus.cc:3055] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:18.947326 27100 raft_consensus.cc:740] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:18.948109 27100 consensus_queue.cc:260] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:18.949203 27100 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:18.957508 27100 raft_consensus.cc:2468] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:47:18.958863 29780 tablet_bootstrap.cc:492] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:18.959338 29782 tablet_copy_client.cc:323] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.1:35499
I20260807 01:47:18.961817 27287 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:18.963927 27337 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:18.961509 27127 tablet_copy_service.cc:140] P 3262a9d183914b699227fdba0ca5bed3: Received BeginTabletCopySession request for tablet 31708844dc4f41feb6a7a2820cfc0e0b from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:52556)
I20260807 01:47:18.964684 27127 tablet_copy_service.cc:161] P 3262a9d183914b699227fdba0ca5bed3: Beginning new tablet copy session on tablet 31708844dc4f41feb6a7a2820cfc0e0b from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:52556: session id = 57bb4985157447f0a3ab680fa0026f12-31708844dc4f41feb6a7a2820cfc0e0b
I20260807 01:47:18.969607 29658 raft_consensus.cc:2804] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:47:18.970265 29658 raft_consensus.cc:697] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 5 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:18.971156 29658 consensus_queue.cc:237] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:18.986577 27216 raft_consensus.cc:2468] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:47:18.988626 27127 tablet_copy_source_session.cc:215] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:18.992359 29782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31708844dc4f41feb6a7a2820cfc0e0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:19.002295 26949 catalog_manager.cc:5697] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 4 to 5, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 5 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:47:19.020514 29780 tablet_bootstrap.cc:492] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:19.020756 29782 tablet_copy_client.cc:806] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:19.021595 29780 tablet_bootstrap.cc:492] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:19.022509 29780 ts_tablet_manager.cc:1403] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.064s user 0.028s sys 0.004s
I20260807 01:47:19.021538 29782 tablet_copy_client.cc:670] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:19.026008 29780 raft_consensus.cc:359] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:19.026945 29780 raft_consensus.cc:740] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:19.027678 29780 consensus_queue.cc:260] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:19.030261 29780 ts_tablet_manager.cc:1434] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.007s user 0.005s sys 0.003s
I20260807 01:47:19.032285 29782 tablet_copy_client.cc:538] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:19.032686 27246 tablet_copy_service.cc:342] P 3478729505b046c2a0051c2d0130873f: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-565d0a4463ac45e4bad1886cc0095ff7 received from {username='slave'} at 127.0.0.1:41230
I20260807 01:47:19.033159 27246 tablet_copy_service.cc:434] P 3478729505b046c2a0051c2d0130873f: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-565d0a4463ac45e4bad1886cc0095ff7 on tablet 565d0a4463ac45e4bad1886cc0095ff7 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:19.042626 29782 tablet_bootstrap.cc:492] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:19.061969 29710 raft_consensus.cc:993] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:19.062589 29710 raft_consensus.cc:1081] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:19.065665 27100 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:41080
I20260807 01:47:19.066425 27100 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:19.066913 27100 raft_consensus.cc:3060] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:19.076189 27100 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:19.078707 27100 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:19.079180 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:19.079890 27216 raft_consensus.cc:3055] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:19.080250 27216 raft_consensus.cc:740] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:19.080886 29782 tablet_bootstrap.cc:492] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:19.080981 27216 consensus_queue.cc:260] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:19.082073 29782 tablet_bootstrap.cc:492] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:19.082937 29782 ts_tablet_manager.cc:1403] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.041s user 0.027s sys 0.009s
I20260807 01:47:19.083611 27216 raft_consensus.cc:3060] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:19.084312 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:19.084906 27328 raft_consensus.cc:3060] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:19.092856 29782 raft_consensus.cc:359] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:19.093815 29782 raft_consensus.cc:740] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:19.094492 29782 consensus_queue.cc:260] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:19.095610 27328 raft_consensus.cc:2468] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:47:19.096908 29782 ts_tablet_manager.cc:1434] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260807 01:47:19.099341 27063 leader_election.cc:304] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:19.100214 29421 raft_consensus.cc:2804] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:47:19.100674 29421 raft_consensus.cc:697] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:19.101465 29421 consensus_queue.cc:237] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:19.126452 27127 tablet_copy_service.cc:342] P 3262a9d183914b699227fdba0ca5bed3: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-31708844dc4f41feb6a7a2820cfc0e0b received from {username='slave'} at 127.0.0.1:52556
I20260807 01:47:19.126946 27127 tablet_copy_service.cc:434] P 3262a9d183914b699227fdba0ca5bed3: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-31708844dc4f41feb6a7a2820cfc0e0b on tablet 31708844dc4f41feb6a7a2820cfc0e0b with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:19.129647 27216 raft_consensus.cc:2468] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:47:19.137751 26949 catalog_manager.cc:5697] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 2 to 3, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:19.186800 29782 ts_tablet_manager.cc:933] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:19.190107 29782 tablet_copy_client.cc:323] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.3:42429
I20260807 01:47:19.192158 27350 tablet_copy_service.cc:140] P fcf4e45defb94276bcfa68392711dade: Received BeginTabletCopySession request for tablet 08edf043c99f445b99e723211e8e0ad0 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:36604)
I20260807 01:47:19.192775 27350 tablet_copy_service.cc:161] P fcf4e45defb94276bcfa68392711dade: Beginning new tablet copy session on tablet 08edf043c99f445b99e723211e8e0ad0 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:36604: session id = 57bb4985157447f0a3ab680fa0026f12-08edf043c99f445b99e723211e8e0ad0
I20260807 01:47:19.200944 27350 tablet_copy_source_session.cc:215] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:19.204351 29782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08edf043c99f445b99e723211e8e0ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:19.220521 29782 tablet_copy_client.cc:806] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:19.221135 29782 tablet_copy_client.cc:670] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:19.225559 29782 tablet_copy_client.cc:538] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:19.237125 29782 tablet_bootstrap.cc:492] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:19.256844 27103 raft_consensus.cc:1275] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:19.258488 29715 consensus_queue.cc:1048] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:19.266958 29782 tablet_bootstrap.cc:492] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:19.268092 29782 tablet_bootstrap.cc:492] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:19.268966 29782 ts_tablet_manager.cc:1403] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.032s user 0.019s sys 0.011s
I20260807 01:47:19.272217 29782 raft_consensus.cc:359] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:19.273053 29782 raft_consensus.cc:740] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:19.273805 29782 consensus_queue.cc:260] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:19.277113 29782 ts_tablet_manager.cc:1434] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.008s user 0.005s sys 0.005s
I20260807 01:47:19.278983 27350 tablet_copy_service.cc:342] P fcf4e45defb94276bcfa68392711dade: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-08edf043c99f445b99e723211e8e0ad0 received from {username='slave'} at 127.0.0.1:36604
I20260807 01:47:19.279457 27350 tablet_copy_service.cc:434] P fcf4e45defb94276bcfa68392711dade: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-08edf043c99f445b99e723211e8e0ad0 on tablet 08edf043c99f445b99e723211e8e0ad0 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:19.297544 28709 raft_consensus.cc:1217] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.3->[4.4-4.4] Dedup: 4.4->[]
I20260807 01:47:19.340206 27338 raft_consensus.cc:1275] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:19.343063 29746 consensus_queue.cc:1048] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:19.383616 27108 raft_consensus.cc:1275] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 01:47:19.386495 29701 consensus_queue.cc:1048] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:19.424767 27216 raft_consensus.cc:1275] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 01:47:19.426067 29701 consensus_queue.cc:1048] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:19.483343 28709 raft_consensus.cc:1217] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20260807 01:47:19.537953 27216 raft_consensus.cc:1275] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:47:19.539317 29750 consensus_queue.cc:1048] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:19.578495 27338 raft_consensus.cc:1275] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:47:19.581718 29742 consensus_queue.cc:1048] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:19.749397 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:19.790669 28715 raft_consensus.cc:1217] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.3->[4.4-4.4] Dedup: 4.4->[]
I20260807 01:47:19.825115 27338 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
mode: GRACEFUL
new_leader_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:57092
I20260807 01:47:19.825692 27338 raft_consensus.cc:606] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Received request to transfer leadership to TS 3478729505b046c2a0051c2d0130873f
I20260807 01:47:19.828662 27103 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3806d33df8144ca5a662db5461461b43"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:19.829267 27103 raft_consensus.cc:606] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:19.833319 27216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53d8de481420436498aadc9930032da5"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:19.834112 27216 raft_consensus.cc:606] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:19.835490 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260807 01:47:19.836762 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:47:19.837495 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:19.853259 29750 raft_consensus.cc:993] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:19.853780 29421 raft_consensus.cc:1081] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Signalling peer fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:19.856685 27337 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3806d33df8144ca5a662db5461461b43"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:33976
I20260807 01:47:19.857240 27337 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:19.857587 27337 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:19.870934 29801 raft_consensus.cc:1064] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:19.873977 29801 consensus_queue.cc:237] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 2, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:19.875221 27337 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:19.879830 29710 raft_consensus.cc:993] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:19.884296 29710 raft_consensus.cc:1081] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:19.887434 27337 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:19.888543 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:19.888842 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:19.889211 27103 raft_consensus.cc:3055] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:19.889333 27218 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:19.889557 27103 raft_consensus.cc:740] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:19.890465 27103 consensus_queue.cc:260] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:19.891750 27103 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:19.901672 28715 raft_consensus.cc:1275] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 01:47:19.904066 29710 consensus_queue.cc:1048] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:47:19.941365 27100 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53d8de481420436498aadc9930032da5"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:41080
I20260807 01:47:19.944903 27337 raft_consensus.cc:1275] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 01:47:19.966305 27100 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:19.966848 27100 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:19.970623 27218 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:47:19.975652 29710 consensus_queue.cc:1048] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:47:19.979316 27100 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:19.982035 27100 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:19.984987 27287 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:19.992313 29778 raft_consensus.cc:2804] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:19.992868 29778 raft_consensus.cc:697] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:19.993714 29778 consensus_queue.cc:237] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:20.019155 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:20.019894 27337 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:20.010664 27103 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:47:20.020756 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:20.021392 27214 raft_consensus.cc:3055] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:20.021704 27214 raft_consensus.cc:740] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:20.022558 27214 consensus_queue.cc:260] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:20.023957 27214 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:20.026465 29658 raft_consensus.cc:993] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade: : Instructing follower 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:20.027380 29658 raft_consensus.cc:1081] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Signalling peer 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:20.033952 27227 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:57654
I20260807 01:47:20.034721 27227 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:20.035137 27227 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:20.041221 26948 catalog_manager.cc:5697] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 3 to 4, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 4 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:47:20.043147 27214 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:20.045004 27227 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:20.051072 27227 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:20.052930 29802 raft_consensus.cc:2955] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:20.062810 27064 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:20.066902 29750 raft_consensus.cc:2804] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:47:20.067672 29750 raft_consensus.cc:697] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:20.066886 27328 raft_consensus.cc:2955] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:20.067937 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:20.068637 29750 consensus_queue.cc:237] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:20.064791 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:20.070604 27338 raft_consensus.cc:3055] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:20.068523 27100 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:20.070925 27338 raft_consensus.cc:740] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:20.072181 27338 consensus_queue.cc:260] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:20.073415 27338 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:20.082204 29742 raft_consensus.cc:1064] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:20.082597 27100 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:47:20.085242 29742 consensus_queue.cc:237] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:20.090771 27338 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:47:20.091572 27103 raft_consensus.cc:1275] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 01:47:20.072739 27337 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:20.092964 27170 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:20.094128 29801 raft_consensus.cc:2804] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:47:20.094683 29801 raft_consensus.cc:697] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:20.096817 29801 consensus_queue.cc:237] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:20.112133 29746 consensus_queue.cc:1048] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:47:20.115954 26948 catalog_manager.cc:5697] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:20.122543 28709 raft_consensus.cc:2955] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:20.138602 27337 raft_consensus.cc:1275] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 01:47:20.138679 28708 raft_consensus.cc:1275] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 01:47:20.140398 29421 consensus_queue.cc:1048] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:20.142982 29421 consensus_queue.cc:1048] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:20.165830 29832 raft_consensus.cc:2955] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:20.167790 27227 raft_consensus.cc:1275] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 01:47:20.182580 27338 raft_consensus.cc:2955] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:20.185977 26949 catalog_manager.cc:5697] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 2 to 3, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 3 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:20.187644 29750 consensus_queue.cc:1048] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:20.185266 26948 catalog_manager.cc:5697] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 4 to 5, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 5 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:20.183244 27103 raft_consensus.cc:2955] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:20.199347 28708 raft_consensus.cc:2955] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:20.221714 27227 raft_consensus.cc:2955] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:20.232161 26950 catalog_manager.cc:5697] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:20.266749 27105 consensus_queue.cc:237] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:20.273918 27338 raft_consensus.cc:1275] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:20.275066 28708 raft_consensus.cc:1275] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:20.276424 29832 consensus_queue.cc:1048] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:47:20.277132 29421 consensus_queue.cc:1048] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:47:20.289853 27337 raft_consensus.cc:2955] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:20.286206 29421 raft_consensus.cc:2955] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:20.298308 29778 raft_consensus.cc:1064] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:20.301299 28715 raft_consensus.cc:2955] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:20.312661 29778 consensus_queue.cc:237] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 2, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:20.339991 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 31708844dc4f41feb6a7a2820cfc0e0b with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:20.347976 26948 catalog_manager.cc:5697] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 4 to 5, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New cstate: current_term: 2 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:20.362139 28708 raft_consensus.cc:1275] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 01:47:20.364094 27214 raft_consensus.cc:1275] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 01:47:20.367242 27105 raft_consensus.cc:1275] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 01:47:20.369395 29816 consensus_queue.cc:1048] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:20.370780 29658 consensus_queue.cc:1048] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:47:20.376521 29701 consensus_queue.cc:1048] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:20.401290 27199 tablet_service.cc:1558] Processing DeleteTablet for tablet 31708844dc4f41feb6a7a2820cfc0e0b with delete_type TABLET_DATA_TOMBSTONED (TS 3478729505b046c2a0051c2d0130873f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:53780
I20260807 01:47:20.449801 29843 tablet_replica.cc:333] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f: stopping tablet replica
I20260807 01:47:20.451196 27214 raft_consensus.cc:1275] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:20.451653 29843 raft_consensus.cc:2243] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:20.452756 29843 raft_consensus.cc:2272] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:20.458707 29843 ts_tablet_manager.cc:1916] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:20.466806 29701 consensus_queue.cc:1048] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:20.473930 27218 raft_consensus.cc:2955] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:20.464107 29816 raft_consensus.cc:2955] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:20.472781 28708 raft_consensus.cc:2955] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:20.504663 27337 raft_consensus.cc:1275] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 01:47:20.514580 29831 consensus_queue.cc:1048] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:20.532472 26949 catalog_manager.cc:5697] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:20.545333 27105 raft_consensus.cc:1275] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:20.559981 29843 ts_tablet_manager.cc:1929] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260807 01:47:20.565569 29658 consensus_queue.cc:1048] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:20.576491 27337 raft_consensus.cc:1275] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:47:20.591801 27227 raft_consensus.cc:1275] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 01:47:20.582666 29843 log.cc:1199] T 31708844dc4f41feb6a7a2820cfc0e0b P 3478729505b046c2a0051c2d0130873f: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-1-root/wals/31708844dc4f41feb6a7a2820cfc0e0b
I20260807 01:47:20.597368 26932 catalog_manager.cc:5028] TS 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): tablet 31708844dc4f41feb6a7a2820cfc0e0b (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:20.603274 27105 raft_consensus.cc:1275] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:47:20.607867 29801 consensus_queue.cc:1048] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:20.608143 27214 consensus_queue.cc:237] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:20.614311 29710 consensus_queue.cc:1048] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260807 01:47:20.603377 29831 consensus_queue.cc:1048] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260807 01:47:20.672772 27103 raft_consensus.cc:2955] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
W20260807 01:47:20.793509 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 68668.6us
W20260807 01:47:20.802646 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 74104.2us
W20260807 01:47:20.854913 29847 log.cc:927] Time spent T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.073s user 0.001s sys 0.000s
I20260807 01:47:20.987491 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
W20260807 01:47:21.014801 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 187582us
I20260807 01:47:21.056922 28715 raft_consensus.cc:1275] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 01:47:21.085222 27337 consensus_queue.cc:237] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:21.155187 27100 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37478d818c864abc83662e98299718d1"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:21.159432 27100 raft_consensus.cc:606] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:21.162109 29746 consensus_queue.cc:1048] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260807 01:47:21.154611 29874 log.cc:927] Time spent T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.138s user 0.002s sys 0.000s
W20260807 01:47:21.207772 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 85547.8us
I20260807 01:47:21.214970 27227 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54eb75662df043488517f09bff1dfcff"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:59426
W20260807 01:47:21.215435 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 93903.7us
I20260807 01:47:21.215606 27227 raft_consensus.cc:606] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:21.227027 28709 raft_consensus.cc:1275] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 01:47:21.240926 27337 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
mode: GRACEFUL
new_leader_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:57092
I20260807 01:47:21.241588 27337 raft_consensus.cc:606] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Received request to transfer leadership to TS 3478729505b046c2a0051c2d0130873f
I20260807 01:47:21.241671 27100 raft_consensus.cc:1275] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260807 01:47:21.251174 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 98861.5us
I20260807 01:47:21.259061 27105 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:21.259732 27105 raft_consensus.cc:606] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Received request to transfer leadership to TS 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:21.264499 29802 consensus_queue.cc:1048] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 01:47:21.268875 27216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:21.269574 27216 raft_consensus.cc:606] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:21.274580 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260807 01:47:21.276415 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:47:21.277244 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:21.296754 29880 raft_consensus.cc:2955] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:21.308647 29816 raft_consensus.cc:993] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade: : Instructing follower 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:21.309267 29701 raft_consensus.cc:1081] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Signalling peer 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:21.312431 28709 raft_consensus.cc:2955] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:21.317137 27216 raft_consensus.cc:1275] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 01:47:21.320768 27099 raft_consensus.cc:2955] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:21.323843 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 565d0a4463ac45e4bad1886cc0095ff7 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:21.332497 29906 consensus_queue.cc:1048] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:47:21.350796 27216 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:57654
I20260807 01:47:21.351508 27216 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:21.351873 27216 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:21.348500 26950 catalog_manager.cc:5697] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 5 to 6, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New cstate: current_term: 4 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:21.358739 29906 raft_consensus.cc:2955] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:21.360005 29897 raft_consensus.cc:993] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:21.359882 28714 raft_consensus.cc:2955] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:21.360862 29897 raft_consensus.cc:1081] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:21.366246 29899 raft_consensus.cc:993] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:21.366878 29880 raft_consensus.cc:1081] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:21.389859 27103 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:41080
I20260807 01:47:21.390488 27103 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:21.390885 27103 raft_consensus.cc:3060] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:21.394970 29750 raft_consensus.cc:993] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:47:21.395737 29750 raft_consensus.cc:1081] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Signalling peer 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:47:21.401738 27103 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:21.403661 28714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:52322
I20260807 01:47:21.404313 28714 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:21.404688 28714 raft_consensus.cc:3060] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:21.405550 27109 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "54eb75662df043488517f09bff1dfcff"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:41080
I20260807 01:47:21.406255 27109 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:21.406636 27109 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:21.409334 26931 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 08edf043c99f445b99e723211e8e0ad0 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:21.418553 28714 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:21.431655 26949 catalog_manager.cc:5697] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index 5 to 6, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New cstate: current_term: 4 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:21.453852 28714 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:21.460089 27216 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:21.468161 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:21.469095 27227 raft_consensus.cc:3055] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:21.469499 27227 raft_consensus.cc:740] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:21.470686 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:21.471167 27227 consensus_queue.cc:260] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:21.469594 27103 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:21.471333 27099 raft_consensus.cc:3055] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:21.469282 27109 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:21.472767 27099 raft_consensus.cc:740] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:21.474900 27099 consensus_queue.cc:260] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:21.476084 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:21.477305 27099 raft_consensus.cc:3060] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:21.476660 27109 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:21.472232 27227 raft_consensus.cc:3060] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:21.476717 27105 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:21.491818 27099 raft_consensus.cc:2468] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:47:21.494323 27105 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:47:21.502753 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:21.490051 27227 raft_consensus.cc:2468] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:21.503767 27214 raft_consensus.cc:3060] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:21.511772 27218 raft_consensus.cc:1240] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:21.534505 27216 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:21.543951 27214 raft_consensus.cc:2468] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:47:21.551043 28660 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:21.553659 29868 raft_consensus.cc:2804] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:21.554370 29868 raft_consensus.cc:697] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:21.555581 29868 consensus_queue.cc:237] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:21.562505 29906 consensus_queue.cc:1048] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:47:21.604010 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:21.604717 27216 raft_consensus.cc:3055] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:21.605017 27216 raft_consensus.cc:740] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:21.611246 29816 consensus_queue.cc:1059] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:21.612280 27317 tablet_service.cc:1558] Processing DeleteTablet for tablet 565d0a4463ac45e4bad1886cc0095ff7 with delete_type TABLET_DATA_TOMBSTONED (TS fcf4e45defb94276bcfa68392711dade not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:59474
I20260807 01:47:21.613049 29854 raft_consensus.cc:3055] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:21.613490 29854 raft_consensus.cc:740] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:21.614377 29854 consensus_queue.cc:260] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:21.629315 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:21.630916 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:21.631580 27338 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:21.630030 27325 raft_consensus.cc:3060] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:21.635545 29854 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:21.639941 27338 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:47:21.605762 27216 consensus_queue.cc:260] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:21.647437 27216 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:21.653383 27325 raft_consensus.cc:2468] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:21.662518 27216 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:47:21.673112 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:21.678531 27063 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:21.682968 27063 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:21.676151 29778 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:21.680701 29877 raft_consensus.cc:2804] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:21.738719 29877 raft_consensus.cc:697] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
W20260807 01:47:21.757344 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 74194.9us
I20260807 01:47:21.757591 29877 consensus_queue.cc:237] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:21.758282 27172 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:21.738946 29750 raft_consensus.cc:2804] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:47:21.785490 26948 catalog_manager.cc:5697] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 3 to 4, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4). New cstate: current_term: 4 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:21.788064 27227 raft_consensus.cc:2955] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:21.737334 29778 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:21.766371 29746 raft_consensus.cc:2804] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election won for term 3
W20260807 01:47:21.808585 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 125700us
I20260807 01:47:21.792379 29750 raft_consensus.cc:697] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:21.807489 29746 raft_consensus.cc:697] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:21.812013 27088 tablet_service.cc:1558] Processing DeleteTablet for tablet 08edf043c99f445b99e723211e8e0ad0 with delete_type TABLET_DATA_TOMBSTONED (TS 3262a9d183914b699227fdba0ca5bed3 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41078
W20260807 01:47:21.834023 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 131346us
I20260807 01:47:21.857640 27337 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:47:21.860008 29778 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:21.862368 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:21.866729 29750 consensus_queue.cc:237] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:21.869236 29746 consensus_queue.cc:237] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:21.901494 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:47:21.976462 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 91506.6us
I20260807 01:47:21.979784 27287 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:21.989920 29920 tablet_replica.cc:333] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade: stopping tablet replica
W20260807 01:47:21.999018 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 117593us
I20260807 01:47:21.992586 29906 raft_consensus.cc:2749] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260807 01:47:22.063599 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 59006.5us
W20260807 01:47:22.066447 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 79868.8us
W20260807 01:47:22.069919 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 48239.1us
I20260807 01:47:22.039875 29920 raft_consensus.cc:2243] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:22.094702 29932 tablet_replica.cc:333] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3: stopping tablet replica
W20260807 01:47:22.140240 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 140530us
I20260807 01:47:22.158444 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:22.165963 29932 raft_consensus.cc:2243] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:22.169713 27100 raft_consensus.cc:1275] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 01:47:22.174551 29940 consensus_queue.cc:1048] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:47:22.175652 29932 raft_consensus.cc:2272] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:22.182580 29832 raft_consensus.cc:993] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:22.183867 29832 raft_consensus.cc:1081] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Signalling peer fcf4e45defb94276bcfa68392711dade to start an election
W20260807 01:47:22.196604 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 132589us
I20260807 01:47:22.218852 29920 raft_consensus.cc:2272] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:22.242991 27229 raft_consensus.cc:1275] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 01:47:22.249048 29893 consensus_queue.cc:1048] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260807 01:47:22.282619 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 82007.1us
W20260807 01:47:22.299803 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 105166us
I20260807 01:47:22.284456 29932 ts_tablet_manager.cc:1916] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:22.336237 26949 catalog_manager.cc:5697] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 3 to 4, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 4 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:22.340947 26948 catalog_manager.cc:5697] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 2 to 3, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 3 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:22.347556 27337 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "37478d818c864abc83662e98299718d1"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:33976
I20260807 01:47:22.348237 27337 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:22.348518 27337 raft_consensus.cc:3060] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:22.360343 27227 raft_consensus.cc:1275] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:22.366550 27337 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:22.377233 27333 raft_consensus.cc:1275] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:47:22.379120 27337 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:22.381088 26949 catalog_manager.cc:5697] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 2, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:22.397079 27103 raft_consensus.cc:1275] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:47:22.403559 29920 ts_tablet_manager.cc:1916] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:22.405287 27227 raft_consensus.cc:1275] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:22.427223 29873 consensus_queue.cc:1048] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260807 01:47:22.463220 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:22.463935 27103 raft_consensus.cc:3055] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:22.464252 27103 raft_consensus.cc:740] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:22.464965 27103 consensus_queue.cc:260] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:22.472404 29928 consensus_queue.cc:1048] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:47:22.477700 27337 raft_consensus.cc:1275] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:22.478477 27103 raft_consensus.cc:3060] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:22.480150 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:22.481052 27229 raft_consensus.cc:3060] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:22.482551 27336 raft_consensus.cc:1275] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:22.503980 27103 raft_consensus.cc:2468] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:47:22.536078 29928 consensus_queue.cc:1048] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.077s
I20260807 01:47:22.619827 29961 consensus_queue.cc:1048] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260807 01:47:22.621596 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 94978.2us
I20260807 01:47:22.670629 27229 raft_consensus.cc:2468] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:47:22.638439 29944 consensus_queue.cc:1048] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
W20260807 01:47:22.687912 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 184683us
W20260807 01:47:22.705333 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 91805.1us
I20260807 01:47:22.701215 29802 consensus_queue.cc:1048] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
W20260807 01:47:22.717293 29613 log.cc:927] Time spent T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df: Append to log took a long time: real 0.155s user 0.001s sys 0.003s
I20260807 01:47:22.731334 27287 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:22.734130 29855 raft_consensus.cc:2804] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:22.734773 29855 raft_consensus.cc:697] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:22.735270 29932 ts_tablet_manager.cc:1929] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.5
I20260807 01:47:22.735965 29932 log.cc:1199] T 08edf043c99f445b99e723211e8e0ad0 P 3262a9d183914b699227fdba0ca5bed3: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-0-root/wals/08edf043c99f445b99e723211e8e0ad0
I20260807 01:47:22.735707 29855 consensus_queue.cc:237] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:22.740391 26932 catalog_manager.cc:5028] TS 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): tablet 08edf043c99f445b99e723211e8e0ad0 (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
W20260807 01:47:22.782953 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 176359us
I20260807 01:47:22.799602 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
W20260807 01:47:22.806080 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 72173.1us
I20260807 01:47:22.861848 29920 ts_tablet_manager.cc:1929] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.5
I20260807 01:47:22.876952 29920 log.cc:1199] T 565d0a4463ac45e4bad1886cc0095ff7 P fcf4e45defb94276bcfa68392711dade: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-2-root/wals/565d0a4463ac45e4bad1886cc0095ff7
I20260807 01:47:22.927017 29920 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59474 (request call id 75) took 1315 ms. Trace:
I20260807 01:47:22.929782 29920 rpcz_store.cc:276] 0807 01:47:21.611943 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:21.612098 (+ 155us) service_pool.cc:224] Handling call
0807 01:47:21.793648 (+181550us) ts_tablet_manager.cc:1151] Deleting tablet 565d0a4463ac45e4bad1886cc0095ff7
0807 01:47:21.800165 (+ 6517us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0807 01:47:21.989727 (+189562us) spinlock_profiling.cc:243] Waited 4.91 ms on lock 0x7b5400407754. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554273d9 00007f7d5a466292 00007f7d5a4970e0 00007f7d5a47d64d 00007f7d5a47d5f9 00007f7d5a47d589 00007f7d5a47d551 00007f7d5a47c77d 00000000003aa554 00000000003aa399
0807 01:47:22.158226 (+168499us) spinlock_profiling.cc:243] Waited 48.1 ms on lock 0x7b5400407754. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d55426742 00007f7d58e30d2c 00007f7d5a46641a 00007f7d5a4970e0 00007f7d5a47d64d 00007f7d5a47d5f9 00007f7d5a47d589 00007f7d5a47d551 00007f7d5a47c77d 00000000003aa554
0807 01:47:22.785560 (+627334us) tablet_metadata.cc:665] Metadata flushed
0807 01:47:22.854060 (+ 68500us) tablet_metadata.cc:665] Metadata flushed
0807 01:47:22.926915 (+ 72855us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36524,"spinlock_wait_cycles":121943680,"tablet-delete.queue_time_us":170135,"thread_start_us":59876,"threads_started":1}
I20260807 01:47:22.935263 27103 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:22.941228 27103 raft_consensus.cc:606] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:22.943084 26931 catalog_manager.cc:5028] TS fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): tablet 565d0a4463ac45e4bad1886cc0095ff7 (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:22.937990 26948 catalog_manager.cc:5697] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 3 to 4, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 4 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:47:22.944973 27103 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30561911ad3945159907e917cc1bafa7"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:22.945730 27103 raft_consensus.cc:606] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:22.949342 27103 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37478d818c864abc83662e98299718d1"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:22.950537 27103 raft_consensus.cc:606] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Received request to transfer leadership to TS 3478729505b046c2a0051c2d0130873f
I20260807 01:47:22.950970 27103 raft_consensus.cc:614] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260807 01:47:22.953205 24592 auto_leader_rebalancer.cc:467] leader transfer for tablet 37478d818c864abc83662e98299718d1 (from 3262a9d183914b699227fdba0ca5bed3 to 3478729505b046c2a0051c2d0130873f) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 01:47:22.955842 27218 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:22.956467 27218 raft_consensus.cc:606] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:22.958256 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260807 01:47:22.960116 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:47:22.961314 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:22.977424 29875 raft_consensus.cc:993] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:22.979063 29802 raft_consensus.cc:1081] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:22.996513 27108 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:41080
I20260807 01:47:22.997219 27108 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:22.997596 27108 raft_consensus.cc:3060] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:23.011443 27108 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:23.147938 29946 raft_consensus.cc:993] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:23.156076 29949 raft_consensus.cc:1081] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Signalling peer fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:23.163733 27108 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:23.170112 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:23.170925 28708 raft_consensus.cc:3060] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Advancing to term 2
W20260807 01:47:23.191792 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 50277.3us
I20260807 01:47:23.206254 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:23.206995 27214 raft_consensus.cc:3055] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:23.207362 27214 raft_consensus.cc:740] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:23.208374 27214 consensus_queue.cc:260] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:23.209455 27214 raft_consensus.cc:3060] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:23.210120 28708 raft_consensus.cc:2468] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:23.211913 28715 raft_consensus.cc:1240] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 1. Current term is 2. Ops: []
I20260807 01:47:23.212616 27067 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:23.213585 29866 raft_consensus.cc:2804] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:47:23.214183 29866 raft_consensus.cc:697] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:23.215514 29866 consensus_queue.cc:237] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:23.294453 27222 raft_consensus.cc:1275] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:23.283499 27214 raft_consensus.cc:2468] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:23.281572 26949 catalog_manager.cc:5697] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 2, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:23.326792 27108 raft_consensus.cc:1240] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 1. Current term is 2. Ops: []
I20260807 01:47:23.327576 27326 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "30561911ad3945159907e917cc1bafa7"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:33976
I20260807 01:47:23.329329 27326 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:23.329713 27326 raft_consensus.cc:3060] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:23.357342 27326 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:23.375334 29866 raft_consensus.cc:993] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:23.375964 29866 raft_consensus.cc:1081] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Signalling peer fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:23.389490 27109 raft_consensus.cc:1275] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:23.391306 27326 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:23.415459 27109 consensus_queue.cc:237] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:23.437520 29894 consensus_queue.cc:1048] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260807 01:47:23.479894 27067 consensus_peers.cc:597] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 09799289505a4c188931e0f3ed0adfb3. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:23.493626 27214 raft_consensus.cc:1275] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:47:23.505240 29866 consensus_queue.cc:1048] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:23.523844 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:23.524542 27108 raft_consensus.cc:3055] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:23.524837 27108 raft_consensus.cc:740] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:23.525657 27108 consensus_queue.cc:260] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:23.527364 27108 raft_consensus.cc:3060] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:23.536074 27108 raft_consensus.cc:2468] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:47:23.540879 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:23.555682 27214 raft_consensus.cc:3060] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:23.550637 29987 raft_consensus.cc:2955] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:23.569929 27214 raft_consensus.cc:2468] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:47:23.572448 27218 raft_consensus.cc:2955] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:23.597865 27326 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:33976
I20260807 01:47:23.598769 27326 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:23.599176 27326 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:23.604948 27336 raft_consensus.cc:1275] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:47:23.609458 27326 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:23.609118 27108 consensus_queue.cc:237] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:23.610479 27333 raft_consensus.cc:1240] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 3. Current term is 4. Ops: []
I20260807 01:47:23.612340 27218 raft_consensus.cc:1275] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:23.613780 29742 consensus_queue.cc:1048] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:47:23.622833 26949 catalog_manager.cc:5697] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:23.627519 29975 consensus_queue.cc:1048] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:23.631299 27326 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:23.657830 27287 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:23.660115 29973 raft_consensus.cc:2804] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:47:23.660786 28708 raft_consensus.cc:1275] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:23.660837 29973 raft_consensus.cc:697] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:23.663077 29742 consensus_queue.cc:1048] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:47:23.663486 29973 consensus_queue.cc:237] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:23.688831 29878 consensus_queue.cc:1048] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:23.690336 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:23.691080 27109 raft_consensus.cc:3055] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:23.691254 29421 consensus_queue.cc:1059] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:23.691656 27109 raft_consensus.cc:740] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:23.693992 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:23.694662 27222 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:23.696620 27109 consensus_queue.cc:260] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:23.702333 26948 catalog_manager.cc:5697] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 2 to 3, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 3 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:47:23.703420 27222 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:47:23.730389 27222 raft_consensus.cc:1275] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260807 01:47:23.734570 27067 consensus_peers.cc:597] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 53d8de481420436498aadc9930032da5. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:23.736029 29878 consensus_queue.cc:1048] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:23.736316 27287 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:23.738438 29997 raft_consensus.cc:2804] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:23.739006 29997 raft_consensus.cc:697] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:23.740125 29997 consensus_queue.cc:237] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:23.749388 27109 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
W20260807 01:47:23.755581 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 68789.7us
I20260807 01:47:23.788530 27338 raft_consensus.cc:1275] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260807 01:47:23.789062 27109 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
W20260807 01:47:23.894546 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 106614us
W20260807 01:47:23.922772 29999 log.cc:927] Time spent T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.052s user 0.000s sys 0.001s
I20260807 01:47:23.950670 30016 ts_tablet_manager.cc:933] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:47:23.967460 27325 raft_consensus.cc:2955] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:23.988847 26948 catalog_manager.cc:5697] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 3 to 4, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 4 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:47:23.991410 30016 tablet_copy_client.cc:323] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.1:35499
I20260807 01:47:23.965502 29987 consensus_queue.cc:1048] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
W20260807 01:47:24.017226 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 44479.3us
W20260807 01:47:24.045363 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 62402.9us
W20260807 01:47:24.058442 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 79717.4us
I20260807 01:47:24.064254 27127 tablet_copy_service.cc:140] P 3262a9d183914b699227fdba0ca5bed3: Received BeginTabletCopySession request for tablet 09799289505a4c188931e0f3ed0adfb3 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:52556)
I20260807 01:47:24.064874 27127 tablet_copy_service.cc:161] P 3262a9d183914b699227fdba0ca5bed3: Beginning new tablet copy session on tablet 09799289505a4c188931e0f3ed0adfb3 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:52556: session id = 57bb4985157447f0a3ab680fa0026f12-09799289505a4c188931e0f3ed0adfb3
I20260807 01:47:24.075515 27127 tablet_copy_source_session.cc:215] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:24.078435 30006 raft_consensus.cc:2955] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Committing config change with OpId 5.5: config changed from index -1 to 5, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:24.095723 27222 raft_consensus.cc:2955] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index -1 to 5, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
W20260807 01:47:24.099958 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 49557.5us
I20260807 01:47:24.125414 26948 catalog_manager.cc:5697] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index -1 to 5, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 5 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:24.126613 30037 ts_tablet_manager.cc:933] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:47:24.155097 30037 tablet_copy_client.cc:323] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.1:35499
I20260807 01:47:24.211881 27127 tablet_copy_service.cc:140] P 3262a9d183914b699227fdba0ca5bed3: Received BeginTabletCopySession request for tablet 53d8de481420436498aadc9930032da5 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:52556)
I20260807 01:47:24.212534 27127 tablet_copy_service.cc:161] P 3262a9d183914b699227fdba0ca5bed3: Beginning new tablet copy session on tablet 53d8de481420436498aadc9930032da5 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:52556: session id = 57bb4985157447f0a3ab680fa0026f12-53d8de481420436498aadc9930032da5
I20260807 01:47:24.215061 27214 consensus_queue.cc:237] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:24.235297 30016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09799289505a4c188931e0f3ed0adfb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:24.255136 27100 raft_consensus.cc:1275] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:24.277837 27100 raft_consensus.cc:1275] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 01:47:24.280496 30019 consensus_queue.cc:1048] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:47:24.292856 27222 raft_consensus.cc:1275] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:24.293416 30016 tablet_copy_client.cc:806] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:24.313645 30016 tablet_copy_client.cc:670] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:24.319875 27100 raft_consensus.cc:1275] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:47:24.313791 27127 tablet_copy_source_session.cc:215] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:24.332866 30037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53d8de481420436498aadc9930032da5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:24.350970 30016 tablet_copy_client.cc:538] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:24.372718 29973 consensus_queue.cc:1048] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:24.378615 30019 raft_consensus.cc:2955] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:24.382501 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:24.389981 29973 consensus_queue.cc:1048] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:24.392196 27103 raft_consensus.cc:2955] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:24.397818 29973 consensus_queue.cc:1048] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:24.402503 27100 consensus_queue.cc:237] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:24.414318 27222 raft_consensus.cc:1275] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:47:24.425199 30016 tablet_bootstrap.cc:492] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:24.424911 26949 catalog_manager.cc:5697] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:24.440778 27336 raft_consensus.cc:1275] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260807 01:47:24.450419 27067 consensus_peers.cc:597] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9febf660f0fd492dabb2b64043136fbe. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:24.453073 27222 raft_consensus.cc:1275] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 01:47:24.454597 29831 consensus_queue.cc:1048] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260807 01:47:24.483762 27175 consensus_peers.cc:597] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f7a8b225f3fd4b65a1f62d70c6fd2bb7. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:24.489334 27325 raft_consensus.cc:2955] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index -1 to 5, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:24.479578 30037 tablet_copy_client.cc:806] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:24.495464 30037 tablet_copy_client.cc:670] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:24.508740 29866 raft_consensus.cc:2955] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:24.524715 30037 tablet_copy_client.cc:538] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:24.528769 27230 raft_consensus.cc:2955] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:24.530840 27325 raft_consensus.cc:1275] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 01:47:24.552891 30037 tablet_bootstrap.cc:492] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:24.556777 29974 consensus_queue.cc:1048] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:47:24.706826 29987 consensus_queue.cc:1048] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260807 01:47:24.730800 26949 catalog_manager.cc:5697] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index -1 to 4, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:24.741827 29997 consensus_queue.cc:1048] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:24.732791 30016 tablet_bootstrap.cc:492] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:24.752998 30016 tablet_bootstrap.cc:492] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:24.753978 30016 ts_tablet_manager.cc:1403] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.329s user 0.044s sys 0.011s
I20260807 01:47:24.771386 30016 raft_consensus.cc:359] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:24.782336 30016 raft_consensus.cc:740] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:24.783193 30016 consensus_queue.cc:260] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:24.813987 27326 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:57092
I20260807 01:47:24.814679 27326 raft_consensus.cc:606] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:24.819907 27326 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
mode: GRACEFUL
new_leader_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:57092
I20260807 01:47:24.820506 27326 raft_consensus.cc:606] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Received request to transfer leadership to TS 3478729505b046c2a0051c2d0130873f
I20260807 01:47:24.825656 27325 raft_consensus.cc:2955] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:24.848896 27100 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:24.849587 27100 raft_consensus.cc:606] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Received request to transfer leadership to TS 3478729505b046c2a0051c2d0130873f
I20260807 01:47:24.864248 27326 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:57092
I20260807 01:47:24.864948 27326 raft_consensus.cc:606] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:24.864476 30016 ts_tablet_manager.cc:1434] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.110s user 0.001s sys 0.007s
I20260807 01:47:24.867600 30055 raft_consensus.cc:993] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:24.868108 30055 raft_consensus.cc:1081] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:24.868245 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260807 01:47:24.869122 27127 tablet_copy_service.cc:342] P 3262a9d183914b699227fdba0ca5bed3: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-09799289505a4c188931e0f3ed0adfb3 received from {username='slave'} at 127.0.0.1:52556
I20260807 01:47:24.869549 27127 tablet_copy_service.cc:434] P 3262a9d183914b699227fdba0ca5bed3: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-09799289505a4c188931e0f3ed0adfb3 on tablet 09799289505a4c188931e0f3ed0adfb3 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:24.870294 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:47:24.873327 30016 ts_tablet_manager.cc:933] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:24.871145 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:24.948336 28715 raft_consensus.cc:1217] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3] Dedup: 3.3->[]
I20260807 01:47:24.987357 30016 tablet_copy_client.cc:323] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.2:34855
I20260807 01:47:24.991081 27326 raft_consensus.cc:2955] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:25.011409 30037 tablet_bootstrap.cc:492] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:25.012475 30037 tablet_bootstrap.cc:492] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:25.013376 30037 ts_tablet_manager.cc:1403] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.468s user 0.045s sys 0.050s
W20260807 01:47:25.045904 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 92562us
I20260807 01:47:25.065194 27246 tablet_copy_service.cc:140] P 3478729505b046c2a0051c2d0130873f: Received BeginTabletCopySession request for tablet f7a8b225f3fd4b65a1f62d70c6fd2bb7 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:41230)
I20260807 01:47:25.078860 27246 tablet_copy_service.cc:161] P 3478729505b046c2a0051c2d0130873f: Beginning new tablet copy session on tablet f7a8b225f3fd4b65a1f62d70c6fd2bb7 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:41230: session id = 57bb4985157447f0a3ab680fa0026f12-f7a8b225f3fd4b65a1f62d70c6fd2bb7
W20260807 01:47:25.158272 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 111729us
W20260807 01:47:25.253507 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 52072.2us
I20260807 01:47:25.131492 30037 raft_consensus.cc:359] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
W20260807 01:47:25.253365 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 56421.2us
I20260807 01:47:25.274457 27109 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:34468
I20260807 01:47:25.275408 27109 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:25.275928 27109 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:25.257881 27246 tablet_copy_source_session.cc:215] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:25.259327 30037 raft_consensus.cc:740] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:25.337579 30037 consensus_queue.cc:260] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
W20260807 01:47:25.336618 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 44320.9us
I20260807 01:47:25.360864 29866 raft_consensus.cc:993] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:25.361883 29866 raft_consensus.cc:1081] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Signalling peer 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:25.370276 30016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7a8b225f3fd4b65a1f62d70c6fd2bb7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260807 01:47:25.388675 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 75864.1us
W20260807 01:47:25.397027 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 75980.6us
I20260807 01:47:25.412441 27109 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:25.344913 30037 ts_tablet_manager.cc:1434] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.331s user 0.013s sys 0.001s
I20260807 01:47:25.433619 27109 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:25.458643 27127 tablet_copy_service.cc:342] P 3262a9d183914b699227fdba0ca5bed3: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-53d8de481420436498aadc9930032da5 received from {username='slave'} at 127.0.0.1:52556
I20260807 01:47:25.459287 27127 tablet_copy_service.cc:434] P 3262a9d183914b699227fdba0ca5bed3: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-53d8de481420436498aadc9930032da5 on tablet 53d8de481420436498aadc9930032da5 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:25.482693 30006 raft_consensus.cc:1064] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
W20260807 01:47:25.483458 30006 raft_consensus.cc:1068] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3: Unable to promote non-voter 57bb4985157447f0a3ab680fa0026f12: Service unavailable: leader transfer in progress
I20260807 01:47:25.483755 30037 ts_tablet_manager.cc:933] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:47:25.482750 28715 raft_consensus.cc:1217] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEARNER]: Deduplicated request from leader. Original: 5.4->[5.5-5.5] Dedup: 5.5->[]
I20260807 01:47:25.522285 30016 tablet_copy_client.cc:806] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:25.534832 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:25.535708 27214 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:25.538630 27218 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:57662
I20260807 01:47:25.539371 27218 raft_consensus.cc:493] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:25.539778 27218 raft_consensus.cc:3060] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:25.572036 27214 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:47:25.523265 30016 tablet_copy_client.cc:670] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:25.586706 27218 raft_consensus.cc:515] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:25.590896 27212 raft_consensus.cc:1240] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:25.592154 27218 leader_election.cc:290] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:25.599372 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:25.600135 27099 raft_consensus.cc:3055] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:25.600505 27099 raft_consensus.cc:740] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:25.601508 27099 consensus_queue.cc:260] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:25.623976 27099 raft_consensus.cc:3060] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:25.627540 30068 raft_consensus.cc:993] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:25.628207 30068 raft_consensus.cc:1081] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:25.630266 30016 tablet_copy_client.cc:538] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:25.639948 27064 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:25.641014 30006 raft_consensus.cc:2804] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:47:25.641677 30006 raft_consensus.cc:697] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:25.642606 30006 consensus_queue.cc:237] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:25.635565 30037 tablet_copy_client.cc:323] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.1:35499
I20260807 01:47:25.676064 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:25.676800 27336 raft_consensus.cc:3055] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:25.677093 27336 raft_consensus.cc:740] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:25.680559 27336 consensus_queue.cc:260] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:25.681970 27336 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:25.675997 27127 tablet_copy_service.cc:140] P 3262a9d183914b699227fdba0ca5bed3: Received BeginTabletCopySession request for tablet 9febf660f0fd492dabb2b64043136fbe from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:52556)
I20260807 01:47:25.684317 27127 tablet_copy_service.cc:161] P 3262a9d183914b699227fdba0ca5bed3: Beginning new tablet copy session on tablet 9febf660f0fd492dabb2b64043136fbe from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:52556: session id = 57bb4985157447f0a3ab680fa0026f12-9febf660f0fd492dabb2b64043136fbe
I20260807 01:47:25.696627 27099 raft_consensus.cc:2468] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:25.715160 27127 tablet_copy_source_session.cc:215] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:25.723816 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:25.724488 27325 raft_consensus.cc:3060] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:25.729673 27172 leader_election.cc:304] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:25.731263 30081 raft_consensus.cc:2804] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:25.731886 30081 raft_consensus.cc:697] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:25.730022 27110 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:34468
I20260807 01:47:25.734573 27110 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:25.734931 27110 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:25.732955 30081 consensus_queue.cc:237] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:25.752601 27110 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:25.757742 27336 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:47:25.760011 30037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9febf660f0fd492dabb2b64043136fbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:25.761842 30016 tablet_bootstrap.cc:492] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:25.807487 27325 raft_consensus.cc:2468] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:25.809918 26948 catalog_manager.cc:5697] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 2 to 3, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:25.819799 30068 raft_consensus.cc:993] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade: : Instructing follower 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:25.820644 30055 raft_consensus.cc:1081] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Signalling peer 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:25.834329 27110 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:25.839781 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:25.840420 27325 raft_consensus.cc:3055] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:25.840740 27325 raft_consensus.cc:740] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:25.841223 27214 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:57654
I20260807 01:47:25.841979 27214 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:25.842465 27214 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:25.841631 27325 consensus_queue.cc:260] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:25.844211 27325 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:25.841327 29987 raft_consensus.cc:1064] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:25.842254 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:25.850900 28708 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:25.848999 29987 consensus_queue.cc:237] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 3, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:25.852417 26949 catalog_manager.cc:5697] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 3 to 4, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 4 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:25.858273 27214 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:25.862020 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:25.862689 27336 raft_consensus.cc:3055] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:25.862998 27336 raft_consensus.cc:740] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:25.863868 27336 consensus_queue.cc:260] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:25.865020 27336 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:25.875070 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:25.875982 28715 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:25.895820 28708 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:47:25.897464 27067 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:25.900501 28715 raft_consensus.cc:2468] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:47:25.911944 27175 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:25.901065 27214 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:25.912931 30081 raft_consensus.cc:2804] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:47:25.913453 30081 raft_consensus.cc:697] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:25.914404 30081 consensus_queue.cc:237] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:25.914925 28708 raft_consensus.cc:1275] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEARNER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 01:47:25.903898 29866 raft_consensus.cc:2804] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:25.898628 30037 tablet_copy_client.cc:806] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:25.906426 27336 raft_consensus.cc:2468] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:47:25.908468 27325 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:47:25.932191 29866 raft_consensus.cc:697] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:25.932772 30087 consensus_queue.cc:1048] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260807 01:47:25.933321 29866 consensus_queue.cc:237] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:25.934702 27338 raft_consensus.cc:1275] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 01:47:25.936874 30037 tablet_copy_client.cc:670] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:25.938977 27214 raft_consensus.cc:1275] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 01:47:25.942126 30006 consensus_queue.cc:1048] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:47:25.963893 30099 consensus_queue.cc:1048] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260807 01:47:25.960486 30037 tablet_copy_client.cc:538] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:26.087586 30016 tablet_bootstrap.cc:492] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:26.096280 30016 tablet_bootstrap.cc:492] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:26.146617 30016 ts_tablet_manager.cc:1403] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.385s user 0.039s sys 0.024s
I20260807 01:47:26.165545 27331 raft_consensus.cc:1275] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:47:26.172353 30037 tablet_bootstrap.cc:492] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:26.171919 30016 raft_consensus.cc:359] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:26.172871 30016 raft_consensus.cc:740] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:26.193615 30087 consensus_queue.cc:1048] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260807 01:47:26.199873 27214 raft_consensus.cc:2955] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 5 to 6, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:26.200479 29866 raft_consensus.cc:2955] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Committing config change with OpId 5.6: config changed from index 5 to 6, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:26.173620 30016 consensus_queue.cc:260] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:26.211489 27227 raft_consensus.cc:1275] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:47:26.216810 30016 ts_tablet_manager.cc:1434] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.065s user 0.008s sys 0.000s
I20260807 01:47:26.220734 27246 tablet_copy_service.cc:342] P 3478729505b046c2a0051c2d0130873f: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-f7a8b225f3fd4b65a1f62d70c6fd2bb7 received from {username='slave'} at 127.0.0.1:41230
I20260807 01:47:26.221257 27246 tablet_copy_service.cc:434] P 3478729505b046c2a0051c2d0130873f: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-f7a8b225f3fd4b65a1f62d70c6fd2bb7 on tablet f7a8b225f3fd4b65a1f62d70c6fd2bb7 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:26.226523 30006 consensus_queue.cc:1048] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:26.239259 27338 raft_consensus.cc:1275] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260807 01:47:26.315585 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 83650.2us
I20260807 01:47:26.310941 27099 raft_consensus.cc:1275] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:47:26.271420 28708 raft_consensus.cc:1217] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260807 01:47:26.302091 26949 catalog_manager.cc:5697] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 4 to 5, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 5 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:26.341084 28714 raft_consensus.cc:2955] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 5 to 6, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:26.349915 26948 catalog_manager.cc:5697] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 3 to 4, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 4 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:26.363158 28715 raft_consensus.cc:3060] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Advancing to term 4
I20260807 01:47:26.352090 26948 catalog_manager.cc:5697] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index 5 to 6, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 5 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260807 01:47:26.374809 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 55575.1us
I20260807 01:47:26.385942 27337 raft_consensus.cc:2955] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 5 to 6, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:26.388324 29974 consensus_queue.cc:1048] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260807 01:47:26.391937 29802 consensus_queue.cc:1048] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:26.413787 28715 raft_consensus.cc:1275] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:47:26.423570 27333 raft_consensus.cc:1275] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260807 01:47:26.431553 30114 log.cc:927] Time spent T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.052s user 0.000s sys 0.000s
I20260807 01:47:26.465994 28715 raft_consensus.cc:1275] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260807 01:47:26.519775 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 141348us
I20260807 01:47:26.522109 28708 raft_consensus.cc:1275] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 01:47:26.541086 30087 consensus_queue.cc:1048] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
W20260807 01:47:26.590476 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 71908.3us
W20260807 01:47:26.611106 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 102813us
W20260807 01:47:26.612633 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 96890.3us
W20260807 01:47:26.620523 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 101964us
I20260807 01:47:26.638968 27338 raft_consensus.cc:1275] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 01:47:26.634518 29974 consensus_queue.cc:1048] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260807 01:47:26.633703 30150 consensus_queue.cc:1048] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
W20260807 01:47:26.716683 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 73974.2us
I20260807 01:47:26.643296 30145 consensus_queue.cc:1048] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
W20260807 01:47:26.767439 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 124326us
W20260807 01:47:26.831300 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 119517us
W20260807 01:47:26.843564 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 115107us
W20260807 01:47:26.874886 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 90959.8us
W20260807 01:47:26.886459 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 169167us
W20260807 01:47:26.898433 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 182963us
W20260807 01:47:26.940805 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 57926.6us
W20260807 01:47:26.969555 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 200730us
W20260807 01:47:27.043619 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 100338us
I20260807 01:47:27.104153 30037 tablet_bootstrap.cc:492] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260807 01:47:27.179636 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 114174us
W20260807 01:47:27.193380 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 149019us
I20260807 01:47:27.192672 30037 tablet_bootstrap.cc:492] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
W20260807 01:47:27.206498 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:47:27.256007 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 54224.9us
W20260807 01:47:27.278046 30191 log.cc:927] Time spent T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.069s user 0.001s sys 0.000s
I20260807 01:47:27.296912 30037 ts_tablet_manager.cc:1403] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 1.125s user 0.041s sys 0.053s
W20260807 01:47:27.428331 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:27.456270 30147 raft_consensus.cc:1064] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:27.459662 30174 raft_consensus.cc:493] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:27.460233 30147 consensus_queue.cc:237] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:27.460309 30174 raft_consensus.cc:515] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:27.463665 30174 leader_election.cc:290] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:27.471012 30037 raft_consensus.cc:359] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:27.472240 30037 raft_consensus.cc:740] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:27.473048 30037 consensus_queue.cc:260] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:27.471118 27110 consensus_queue.cc:237] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:27.475769 30037 ts_tablet_manager.cc:1434] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.140s user 0.002s sys 0.006s
W20260807 01:47:27.484889 30159 log.cc:927] Time spent T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.650s user 0.001s sys 0.012s
I20260807 01:47:27.503597 30146 raft_consensus.cc:1064] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:27.509377 30146 consensus_queue.cc:237] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:27.514223 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:27.525190 27127 tablet_copy_service.cc:342] P 3262a9d183914b699227fdba0ca5bed3: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-9febf660f0fd492dabb2b64043136fbe received from {username='slave'} at 127.0.0.1:52556
I20260807 01:47:27.525771 27127 tablet_copy_service.cc:434] P 3262a9d183914b699227fdba0ca5bed3: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-9febf660f0fd492dabb2b64043136fbe on tablet 9febf660f0fd492dabb2b64043136fbe with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:27.526520 27100 raft_consensus.cc:1275] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 01:47:27.520869 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:27.540366 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:27.570384 30146 consensus_queue.cc:1048] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260807 01:47:27.579552 28715 raft_consensus.cc:1275] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260807 01:47:27.609606 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 48954.1us
I20260807 01:47:27.594995 30099 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:27.662953 30099 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:27.669229 28709 raft_consensus.cc:1275] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:47:27.669803 30172 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
W20260807 01:47:27.621400 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 47041us
I20260807 01:47:27.670444 30172 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:27.672834 30172 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:27.642900 30077 consensus_queue.cc:1048] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.131s
I20260807 01:47:27.692200 30170 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:27.692862 30170 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:27.693081 29802 consensus_queue.cc:1048] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:27.695806 28709 raft_consensus.cc:1275] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 01:47:27.697101 29802 consensus_queue.cc:1048] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260807 01:47:27.710543 28715 raft_consensus.cc:1217] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4] Dedup: 3.4->[]
W20260807 01:47:27.717280 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 49038.7us
W20260807 01:47:27.724007 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 114651us
I20260807 01:47:27.728404 30145 consensus_queue.cc:1048] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:47:27.728469 30136 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:27.730803 30170 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:27.731187 30136 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:27.710361 30077 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:27.732496 30077 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:27.733413 30136 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:27.735011 30099 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:27.735391 30077 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:27.711190 30172 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:27.737293 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:27.738368 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:27.762593 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:27.766669 26950 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46152 (request call id 319) took 1006 ms. Trace:
I20260807 01:47:27.757746 30172 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:27.767189 26950 rpcz_store.cc:276] 0807 01:47:26.744216 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:26.744440 (+ 224us) service_pool.cc:224] Handling call
0807 01:47:27.750329 (+1005889us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":590605,"reported_tablets":3}
I20260807 01:47:27.788971 30172 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:27.790542 27287 leader_election.cc:304] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:27.793368 30172 raft_consensus.cc:2749] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:47:27.798784 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:27.823168 27337 raft_consensus.cc:1275] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 01:47:27.824618 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:27.838665 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:27.841684 27337 raft_consensus.cc:1275] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:47:27.844507 27331 raft_consensus.cc:1275] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 01:47:27.858086 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:27.838765 27063 leader_election.cc:304] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:27.865556 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:27.869371 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:47:27.875368 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:27.879069 27100 raft_consensus.cc:1275] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 01:47:27.884850 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:27.896354 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:27.900669 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:27.900895 30201 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:27.901551 30201 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:27.879365 30099 raft_consensus.cc:2749] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:47:27.910761 30204 raft_consensus.cc:2955] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 3 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:27.924264 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:27.925110 27336 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:47:27.939551 30170 raft_consensus.cc:2955] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Committing config change with OpId 5.7: config changed from index 6 to 7, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
W20260807 01:47:27.947849 30194 log.cc:927] Time spent T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.060s user 0.000s sys 0.000s
I20260807 01:47:27.969183 27063 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:27.966905 27109 raft_consensus.cc:2955] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 3 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:27.977547 30087 raft_consensus.cc:2749] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:27.988027 30201 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:27.968029 27172 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:27.990619 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:27.991361 30165 raft_consensus.cc:2749] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:47:27.992105 30162 consensus_queue.cc:1048] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:27.992702 30166 consensus_queue.cc:1048] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.020s
I20260807 01:47:27.996384 28715 raft_consensus.cc:2955] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Committing config change with OpId 5.7: config changed from index 6 to 7, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:28.009385 27063 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:28.017776 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:27.948064 29987 consensus_queue.cc:1048] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260807 01:47:28.025815 30153 consensus_queue.cc:1048] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260807 01:47:28.030073 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:28.038101 30151 raft_consensus.cc:2749] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:28.041108 27287 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:28.043695 30068 raft_consensus.cc:2749] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:47:28.047729 27287 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:28.047814 28710 raft_consensus.cc:2955] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 3 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:28.054397 30201 raft_consensus.cc:2749] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:28.056404 27287 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:28.077490 30170 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:41078 (request call id 76) took 1241 ms. Trace:
I20260807 01:47:28.091413 26949 catalog_manager.cc:5697] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index 3 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:28.108982 30170 rpcz_store.cc:276] 0807 01:47:26.884359 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:26.884506 (+ 147us) service_pool.cc:224] Handling call
0807 01:47:27.470220 (+585714us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b58005beed0. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5537a48a 00007f7d553fc827 00007f7d55419aea 00007f7d5541ab4a 00007f7d55419cfb 00007f7d5542541c 00007f7d55424a93 00007f7d5541cdda 00007f7d5a4162d9 00007f7d53960db4
0807 01:47:27.477453 (+ 7233us) log.cc:844] Serialized 480 byte log entry
0807 01:47:27.581882 (+104429us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b1800218080. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d5b233f31 00007f7d5b2406d1 00007f7d52291d1a 00007f7d5ba255d9 00007f7d5231a3a0 00007f7d52314e94 00007f7d52314e05 00007f7d55377eeb 00007f7d553fcf43 00007f7d55424b32 00007f7d5541cdda
0807 01:47:27.730109 (+148227us) spinlock_profiling.cc:243] Waited 6.61 ms on lock 0x7b540036b120. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d55397522 00007f7d5537828a 00007f7d5537f133 00007f7d5537f0b9 00007f7d5537f049 00007f7d5537f011 00007f7d5537e1dd 00000000003aa554 00000000003aa399 00007f7d52317db5
0807 01:47:28.077405 (+347296us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7204,"raft.queue_time_us":44514,"raft.run_cpu_time_us":2299,"raft.run_wall_time_us":95532,"spinlock_wait_cycles":1101464064,"thread_start_us":65497,"threads_started":1,"wal-append.queue_time_us":161435}
I20260807 01:47:28.113049 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 53d8de481420436498aadc9930032da5 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:28.113152 27218 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:28.113883 27218 raft_consensus.cc:606] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:28.116048 30138 raft_consensus.cc:2749] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260807 01:47:28.117570 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 61259.9us
I20260807 01:47:28.127409 27218 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:28.128656 27218 raft_consensus.cc:606] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:28.132027 27105 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:28.132640 27105 raft_consensus.cc:606] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:28.142088 27104 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4c4340b503246eca370314d2781034a"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:28.142977 27104 raft_consensus.cc:606] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:28.144527 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260807 01:47:28.146425 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:47:28.146528 30140 raft_consensus.cc:993] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:28.147013 30162 raft_consensus.cc:1081] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:28.151141 27104 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:41080
I20260807 01:47:28.151863 27104 raft_consensus.cc:493] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:28.152315 27104 raft_consensus.cc:3060] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:28.201021 27104 raft_consensus.cc:515] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:28.228801 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:28.229562 27222 raft_consensus.cc:3055] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:28.229884 27222 raft_consensus.cc:740] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:28.232950 27104 leader_election.cc:290] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:28.260053 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:28.258633 27222 consensus_queue.cc:260] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:28.275763 27222 raft_consensus.cc:3060] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:28.289279 27328 raft_consensus.cc:2955] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Committing config change with OpId 5.7: config changed from index 6 to 7, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:28.303416 27325 raft_consensus.cc:2955] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 3 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:28.315409 30170 raft_consensus.cc:993] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:28.315990 30170 raft_consensus.cc:1081] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Signalling peer fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:28.322140 30177 raft_consensus.cc:993] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:28.335992 30006 raft_consensus.cc:1064] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:28.339051 30177 raft_consensus.cc:1081] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Signalling peer fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:28.341972 30153 raft_consensus.cc:2955] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:28.343849 30006 consensus_queue.cc:237] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:28.349489 27222 raft_consensus.cc:2468] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:28.359671 27104 raft_consensus.cc:2955] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:28.375201 27064 leader_election.cc:304] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
W20260807 01:47:28.381002 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 134999us
I20260807 01:47:28.384289 30099 raft_consensus.cc:2804] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:47:28.404314 30099 raft_consensus.cc:697] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:28.405601 30099 consensus_queue.cc:237] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:28.420920 27331 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:33976
I20260807 01:47:28.421558 27331 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:28.421983 27331 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:28.423102 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:28.423731 27338 raft_consensus.cc:3060] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:28.425622 27337 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c4c4340b503246eca370314d2781034a"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:33976
I20260807 01:47:28.426322 27337 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:28.426713 27337 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:28.429405 27333 raft_consensus.cc:1275] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 01:47:28.447674 28715 raft_consensus.cc:1275] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 01:47:28.449601 27216 raft_consensus.cc:1275] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 01:47:28.450627 30006 consensus_queue.cc:1048] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:47:28.459753 29987 consensus_queue.cc:1048] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:47:28.465494 27338 raft_consensus.cc:2468] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:28.471771 30099 consensus_queue.cc:1048] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:28.481737 27337 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:28.497109 28710 raft_consensus.cc:2955] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:28.500670 26954 catalog_manager.cc:5697] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index 6 to 7, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New cstate: current_term: 5 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:28.468673 27331 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:28.505836 27326 raft_consensus.cc:1240] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 3. Current term is 4. Ops: []
I20260807 01:47:28.520910 27337 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:28.534679 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:28.535408 27105 raft_consensus.cc:3055] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:28.535676 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:28.535777 27105 raft_consensus.cc:740] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:28.542547 27214 consensus_queue.cc:237] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:28.547160 27222 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:28.550944 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:28.530385 27333 raft_consensus.cc:2955] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:28.553196 27109 raft_consensus.cc:3055] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:28.553550 27109 raft_consensus.cc:740] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:28.552528 27105 consensus_queue.cc:260] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:28.557356 27331 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:28.563063 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:28.563774 27218 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:28.554334 27109 consensus_queue.cc:260] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:28.579883 27109 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:28.581421 27104 raft_consensus.cc:1275] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 01:47:28.581785 27331 raft_consensus.cc:1240] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 4. Current term is 5. Ops: []
I20260807 01:47:28.602041 27105 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:28.610468 30147 consensus_queue.cc:1048] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260807 01:47:28.622251 28709 raft_consensus.cc:1275] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 01:47:28.624840 27218 raft_consensus.cc:2468] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:47:28.626942 30140 consensus_queue.cc:1048] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260807 01:47:28.633859 26949 catalog_manager.cc:5697] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index 2 to 3, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:28.639977 27222 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:47:28.651121 30140 raft_consensus.cc:2955] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:28.654037 28711 raft_consensus.cc:2955] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:28.659422 27199 tablet_service.cc:1558] Processing DeleteTablet for tablet 53d8de481420436498aadc9930032da5 with delete_type TABLET_DATA_TOMBSTONED (TS 3478729505b046c2a0051c2d0130873f not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:53780
I20260807 01:47:28.678120 30171 raft_consensus.cc:2955] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:28.695693 27109 raft_consensus.cc:2468] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:47:28.697355 27222 raft_consensus.cc:2955] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:28.704250 27105 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:47:28.695597 27110 raft_consensus.cc:2955] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:28.712265 27287 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:28.724558 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 09799289505a4c188931e0f3ed0adfb3 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:28.726382 27287 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:28.727612 30232 raft_consensus.cc:2804] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:47:28.728186 30232 raft_consensus.cc:697] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 5 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:28.729283 30232 consensus_queue.cc:237] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:28.729996 30201 raft_consensus.cc:2804] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:28.748374 30201 raft_consensus.cc:697] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:28.749392 30201 consensus_queue.cc:237] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:28.753890 30153 raft_consensus.cc:993] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:28.756837 30153 raft_consensus.cc:1081] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:28.759428 27216 consensus_queue.cc:237] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:28.765733 27337 raft_consensus.cc:2955] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:28.771227 27109 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:41080
I20260807 01:47:28.771821 27109 raft_consensus.cc:493] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:28.772169 27109 raft_consensus.cc:3060] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:28.775628 28711 raft_consensus.cc:1275] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:47:28.776837 26948 catalog_manager.cc:5697] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 2, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:28.780102 27330 raft_consensus.cc:1275] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 01:47:28.783145 30153 consensus_queue.cc:1048] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:28.787133 30204 consensus_queue.cc:1048] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:28.792629 30204 raft_consensus.cc:2955] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:28.804196 27109 raft_consensus.cc:515] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:28.826872 27109 leader_election.cc:290] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:28.828858 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:28.829487 27325 raft_consensus.cc:3060] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:28.840132 27325 raft_consensus.cc:2468] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:28.841511 27063 leader_election.cc:304] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:28.842847 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:28.843511 27218 raft_consensus.cc:3055] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:28.843850 27218 raft_consensus.cc:740] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:28.846760 28711 raft_consensus.cc:2955] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:28.849028 30087 raft_consensus.cc:2804] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:47:28.849608 30087 raft_consensus.cc:697] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:28.850489 30087 consensus_queue.cc:237] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:28.842209 26949 catalog_manager.cc:5697] T 09799289505a4c188931e0f3ed0adfb3 P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index 5 to 6, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New cstate: current_term: 4 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:28.856418 28710 raft_consensus.cc:2955] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:28.873207 27218 consensus_queue.cc:260] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:28.883457 27216 raft_consensus.cc:1275] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:28.903033 27325 raft_consensus.cc:2955] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:28.916958 27218 raft_consensus.cc:3060] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:28.904570 30177 consensus_queue.cc:1048] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260807 01:47:28.950747 30243 tablet_replica.cc:333] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f: stopping tablet replica
I20260807 01:47:28.993484 27218 raft_consensus.cc:2468] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
W20260807 01:47:29.014798 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 113726us
W20260807 01:47:29.198283 30244 log.cc:927] Time spent T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.053s user 0.001s sys 0.000s
W20260807 01:47:29.229882 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 173350us
W20260807 01:47:29.277603 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 227538us
W20260807 01:47:29.287195 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 363236us
W20260807 01:47:29.324218 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 53683us
W20260807 01:47:29.345774 30204 consensus_meta.cc:287] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.521s user 0.011s sys 0.002s
I20260807 01:47:29.395641 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f7a8b225f3fd4b65a1f62d70c6fd2bb7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260807 01:47:29.447288 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 115382us
I20260807 01:47:29.496538 27218 raft_consensus.cc:1275] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260807 01:47:29.509114 27216 raft_consensus.cc:1275] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:29.518006 27222 raft_consensus.cc:1275] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:29.535084 27317 tablet_service.cc:1558] Processing DeleteTablet for tablet 09799289505a4c188931e0f3ed0adfb3 with delete_type TABLET_DATA_TOMBSTONED (TS fcf4e45defb94276bcfa68392711dade not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:59474
I20260807 01:47:29.541163 30282 tablet_replica.cc:333] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade: stopping tablet replica
I20260807 01:47:29.542449 30282 raft_consensus.cc:2243] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:29.548766 26950 catalog_manager.cc:5697] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index 3 to 4, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:29.551433 26950 catalog_manager.cc:5697] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 4 to 5, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 5 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:47:29.553301 27109 raft_consensus.cc:1275] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260807 01:47:29.575464 30282 raft_consensus.cc:2272] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:29.564370 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:29.581794 30282 ts_tablet_manager.cc:1916] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:29.583470 26950 catalog_manager.cc:5697] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 3 to 4, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 4 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:47:29.582897 30243 raft_consensus.cc:2243] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:29.589217 30243 raft_consensus.cc:2272] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:29.592708 30243 ts_tablet_manager.cc:1916] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:29.630182 30282 ts_tablet_manager.cc:1929] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.5
I20260807 01:47:29.638479 30282 log.cc:1199] T 09799289505a4c188931e0f3ed0adfb3 P fcf4e45defb94276bcfa68392711dade: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-2-root/wals/09799289505a4c188931e0f3ed0adfb3
I20260807 01:47:29.638291 30171 consensus_queue.cc:1048] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:47:29.641189 26931 catalog_manager.cc:5028] TS fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): tablet 09799289505a4c188931e0f3ed0adfb3 (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:29.649586 30201 consensus_queue.cc:1048] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:29.667102 30161 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:29.668179 30161 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:29.677223 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:29.676055 30161 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:29.689631 30260 consensus_queue.cc:1048] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260807 01:47:29.698807 30161 consensus_queue.cc:1048] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260807 01:47:29.718786 27330 raft_consensus.cc:1275] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:29.732676 27328 raft_consensus.cc:1275] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:29.766373 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:29.821280 30006 consensus_queue.cc:1048] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:47:29.880717 30151 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:29.881474 30151 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:29.904374 30151 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:47:29.909543 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 73548.5us
I20260807 01:47:29.914381 27287 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:29.915351 30161 raft_consensus.cc:2749] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260807 01:47:29.907527 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 83118.2us
I20260807 01:47:29.909065 30243 ts_tablet_manager.cc:1929] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 5.6
I20260807 01:47:29.918493 30243 log.cc:1199] T 53d8de481420436498aadc9930032da5 P 3478729505b046c2a0051c2d0130873f: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-1-root/wals/53d8de481420436498aadc9930032da5
I20260807 01:47:29.919893 30243 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53780 (request call id 80) took 1261 ms. Trace:
I20260807 01:47:29.920313 30243 rpcz_store.cc:276] 0807 01:47:28.658871 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:28.659123 (+ 252us) service_pool.cc:224] Handling call
0807 01:47:28.950326 (+291203us) ts_tablet_manager.cc:1151] Deleting tablet 53d8de481420436498aadc9930032da5
0807 01:47:28.950434 (+ 108us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0807 01:47:29.702290 (+751856us) tablet_metadata.cc:665] Metadata flushed
0807 01:47:29.908915 (+206625us) tablet_metadata.cc:665] Metadata flushed
0807 01:47:29.919801 (+ 10886us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":552936,"tablet-delete.queue_time_us":290144,"thread_start_us":212606,"threads_started":1}
I20260807 01:47:29.926023 26932 catalog_manager.cc:5028] TS 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): tablet 53d8de481420436498aadc9930032da5 (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:29.929155 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:29.956264 26954 catalog_manager.cc:5697] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:29.986829 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:47:29.989241 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 85545.8us
I20260807 01:47:30.020426 27067 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:30.070525 27088 tablet_service.cc:1558] Processing DeleteTablet for tablet f7a8b225f3fd4b65a1f62d70c6fd2bb7 with delete_type TABLET_DATA_TOMBSTONED (TS 3262a9d183914b699227fdba0ca5bed3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41078
I20260807 01:47:30.067730 26948 catalog_manager.cc:5697] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 2, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:30.038231 30266 raft_consensus.cc:2749] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:47:30.095103 27109 raft_consensus.cc:1275] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:30.105917 30303 tablet_replica.cc:333] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3: stopping tablet replica
I20260807 01:47:30.082439 29987 consensus_queue.cc:1048] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
W20260807 01:47:30.128682 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 53161.8us
W20260807 01:47:30.132699 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 96949.1us
I20260807 01:47:30.140260 30303 raft_consensus.cc:2243] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:30.141603 27222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:30.142266 27222 raft_consensus.cc:606] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:30.150308 30303 raft_consensus.cc:2272] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:30.153610 30303 ts_tablet_manager.cc:1916] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:30.150401 30161 consensus_queue.cc:1048] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260807 01:47:30.183161 27110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53d8de481420436498aadc9930032da5"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:30.183918 27110 raft_consensus.cc:606] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:30.183648 27109 consensus_queue.cc:237] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:30.190490 27110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:30.191659 27110 raft_consensus.cc:606] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Received request to transfer leadership to TS 3478729505b046c2a0051c2d0130873f
I20260807 01:47:30.202431 27110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:30.203065 27110 raft_consensus.cc:606] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Received request to transfer leadership to TS 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:30.205021 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:30.204541 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260807 01:47:30.210067 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:47:30.214007 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:30.240698 30305 raft_consensus.cc:993] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:47:30.242336 30087 raft_consensus.cc:1081] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Signalling peer 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:47:30.246644 28715 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:52322
I20260807 01:47:30.248283 28715 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:30.248796 28715 raft_consensus.cc:3060] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:30.266034 28711 raft_consensus.cc:1275] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260807 01:47:30.269061 30087 consensus_queue.cc:1048] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260807 01:47:30.292219 28715 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:30.311687 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:30.312441 27222 raft_consensus.cc:3060] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:30.314291 28715 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:30.344859 30151 raft_consensus.cc:993] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:30.345813 30151 raft_consensus.cc:1081] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Signalling peer fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:30.340346 30099 raft_consensus.cc:2955] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:30.350138 28715 raft_consensus.cc:2955] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:30.356739 27328 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53d8de481420436498aadc9930032da5"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:33976
W20260807 01:47:30.391434 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 70722.9us
I20260807 01:47:30.413868 27328 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:30.420660 27328 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:30.423673 26954 catalog_manager.cc:5697] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 5 to 6, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:30.405082 27222 raft_consensus.cc:2468] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:47:30.413111 30303 ts_tablet_manager.cc:1929] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 01:47:30.436374 27328 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:30.443595 27328 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:30.438827 30303 log.cc:1199] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3262a9d183914b699227fdba0ca5bed3: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-0-root/wals/f7a8b225f3fd4b65a1f62d70c6fd2bb7
I20260807 01:47:30.446444 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:30.447322 28660 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:30.456826 26932 catalog_manager.cc:5028] TS 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): tablet f7a8b225f3fd4b65a1f62d70c6fd2bb7 (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:30.457094 28711 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:30.462580 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:30.463265 27105 raft_consensus.cc:3055] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:47:30.463588 27105 raft_consensus.cc:740] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:30.464584 27105 consensus_queue.cc:260] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:30.466531 27105 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:30.468905 30276 raft_consensus.cc:2804] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:47:30.469816 30276 raft_consensus.cc:697] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:30.470948 27216 raft_consensus.cc:1275] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260807 01:47:30.478907 28711 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:47:30.481742 27289 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
W20260807 01:47:30.487530 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 49085.8us
I20260807 01:47:30.491961 30201 raft_consensus.cc:2804] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:47:30.503883 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:30.504704 27099 raft_consensus.cc:3055] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:30.505007 27099 raft_consensus.cc:740] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:30.505837 27099 consensus_queue.cc:260] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:30.511533 27099 raft_consensus.cc:3060] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:30.495144 30201 raft_consensus.cc:697] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:30.519577 27105 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:47:30.520974 30276 consensus_queue.cc:237] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:30.524855 30201 consensus_queue.cc:237] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:30.538345 30006 raft_consensus.cc:993] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:30.538964 30006 raft_consensus.cc:1081] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Signalling peer 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:30.545423 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9febf660f0fd492dabb2b64043136fbe with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:30.574640 27317 tablet_service.cc:1558] Processing DeleteTablet for tablet 9febf660f0fd492dabb2b64043136fbe with delete_type TABLET_DATA_TOMBSTONED (TS fcf4e45defb94276bcfa68392711dade not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:59474
I20260807 01:47:30.588945 27227 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:57662
I20260807 01:47:30.589684 27227 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:30.590071 27227 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
W20260807 01:47:30.593520 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 94485.6us
I20260807 01:47:30.607583 27099 raft_consensus.cc:2468] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:47:30.648679 30327 tablet_replica.cc:333] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade: stopping tablet replica
I20260807 01:47:30.655050 30327 raft_consensus.cc:2243] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:30.616425 30087 consensus_queue.cc:1048] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260807 01:47:30.689543 26954 catalog_manager.cc:5697] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 5 to 6, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 6 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:30.692260 26948 catalog_manager.cc:5697] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 2 to 3, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4). New cstate: current_term: 3 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:30.694877 27227 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:30.697860 27227 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:30.700608 30327 raft_consensus.cc:2272] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Raft consensus is shut down!
W20260807 01:47:30.715198 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 125059us
I20260807 01:47:30.725775 27222 raft_consensus.cc:1240] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 4. Current term is 5. Ops: []
I20260807 01:47:30.717823 30327 ts_tablet_manager.cc:1916] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:30.753962 29987 consensus_queue.cc:1059] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:47:30.808343 30087 raft_consensus.cc:3055] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:30.809037 30087 raft_consensus.cc:740] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:30.811771 30087 consensus_queue.cc:260] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:30.813155 30331 raft_consensus.cc:993] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f: : Instructing follower fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:30.814036 30261 raft_consensus.cc:1081] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Signalling peer fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:30.821436 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:30.833060 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:30.833802 28715 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:30.814234 30087 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:30.862318 28715 raft_consensus.cc:2468] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:47:30.877503 27175 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:30.878640 30264 raft_consensus.cc:2804] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:47:30.879536 30264 raft_consensus.cc:697] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:30.881328 30264 consensus_queue.cc:237] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:30.903246 27216 raft_consensus.cc:2955] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:30.920696 27214 raft_consensus.cc:1275] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260807 01:47:30.926448 27109 raft_consensus.cc:2468] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:47:30.938587 30328 consensus_queue.cc:1048] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260807 01:47:30.955154 30327 ts_tablet_manager.cc:1929] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260807 01:47:30.955732 30327 log.cc:1199] T 9febf660f0fd492dabb2b64043136fbe P fcf4e45defb94276bcfa68392711dade: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-2-root/wals/9febf660f0fd492dabb2b64043136fbe
I20260807 01:47:30.958451 26931 catalog_manager.cc:5028] TS fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): tablet 9febf660f0fd492dabb2b64043136fbe (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:30.960594 27337 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:59490
I20260807 01:47:30.961884 27337 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:30.962308 27337 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:30.972723 27337 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:30.976191 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:30.976821 28715 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:30.988391 27100 raft_consensus.cc:1275] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260807 01:47:31.002305 28715 raft_consensus.cc:2468] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:47:31.003763 27289 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:31.004741 30315 raft_consensus.cc:2804] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:47:31.005270 30315 raft_consensus.cc:697] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:30.967674 26948 catalog_manager.cc:5697] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 4 to 5, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 5 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:31.006575 30315 consensus_queue.cc:237] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:31.010972 27337 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:31.035391 27100 raft_consensus.cc:1275] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260807 01:47:31.035679 30315 consensus_queue.cc:1048] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260807 01:47:31.037323 30122 consensus_queue.cc:1048] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 01:47:31.047225 26954 catalog_manager.cc:5697] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 5 to 6, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 6 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:31.054689 28715 raft_consensus.cc:1275] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260807 01:47:31.061347 30161 consensus_queue.cc:1048] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260807 01:47:31.093417 28711 raft_consensus.cc:1240] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6. Ops: []
I20260807 01:47:31.096170 30264 consensus_queue.cc:1059] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 01:47:31.097450 30264 raft_consensus.cc:3055] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:47:31.098253 30264 raft_consensus.cc:740] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:31.100829 30264 consensus_queue.cc:260] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:31.111665 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:31.108349 30264 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:31.160197 27216 raft_consensus.cc:2468] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:47:31.396778 27109 raft_consensus.cc:1275] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 01:47:31.398486 30264 consensus_queue.cc:1048] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 01:47:31.406988 27227 raft_consensus.cc:1275] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260807 01:47:31.414346 30148 consensus_queue.cc:1048] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260807 01:47:31.458981 28711 raft_consensus.cc:1275] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 01:47:31.461596 30264 consensus_queue.cc:1048] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:47:31.495702 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:31.538918 28715 raft_consensus.cc:1275] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260807 01:47:31.558252 30314 consensus_queue.cc:1048] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260807 01:47:31.647780 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 55141.5us
I20260807 01:47:31.649603 27337 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
mode: GRACEFUL
new_leader_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:57092
I20260807 01:47:31.650287 27337 raft_consensus.cc:606] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Received request to transfer leadership to TS 3478729505b046c2a0051c2d0130873f
I20260807 01:47:31.668257 27330 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:57092
I20260807 01:47:31.668905 27330 raft_consensus.cc:606] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:31.683135 27330 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0e280c976844349b4874a479aa08de2"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:57092
I20260807 01:47:31.683929 27330 raft_consensus.cc:606] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:31.698887 27330 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
mode: GRACEFUL
new_leader_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:57092
I20260807 01:47:31.699518 27330 raft_consensus.cc:606] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Received request to transfer leadership to TS 3478729505b046c2a0051c2d0130873f
I20260807 01:47:31.701098 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260807 01:47:31.703760 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:47:31.705308 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:31.783244 30349 raft_consensus.cc:993] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:31.783999 30161 raft_consensus.cc:1081] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:31.786067 27109 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:34468
I20260807 01:47:31.786736 27109 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:31.792719 27109 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:31.838929 27109 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:31.847725 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:31.848727 27227 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:31.863829 27227 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:31.863878 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:31.865077 27330 raft_consensus.cc:3055] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:31.865420 27330 raft_consensus.cc:740] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:31.866693 27330 consensus_queue.cc:260] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:31.868621 27330 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:31.864472 27109 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:31.876588 27064 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:31.877696 30087 raft_consensus.cc:2804] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:47:31.878252 30087 raft_consensus.cc:697] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:31.879096 30087 consensus_queue.cc:237] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:31.879851 27330 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:31.898407 27337 consensus_queue.cc:237] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
W20260807 01:47:31.924477 27289 consensus_peers.cc:597] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aff03a52fbfd498ab6d04a579e96fd0e. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:31.928893 27218 raft_consensus.cc:1275] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260807 01:47:31.934022 30161 consensus_queue.cc:1048] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:47:31.948004 27105 raft_consensus.cc:1275] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260807 01:47:31.954608 30349 consensus_queue.cc:1048] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 01:47:31.984542 30161 raft_consensus.cc:2955] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Committing config change with OpId 6.6: config changed from index -1 to 6, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:31.965153 26954 catalog_manager.cc:5697] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 4 to 5, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 5 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:32.014510 27218 raft_consensus.cc:2955] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Committing config change with OpId 6.6: config changed from index -1 to 6, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:32.050628 30226 raft_consensus.cc:993] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade: : Instructing follower 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:32.052054 30226 raft_consensus.cc:1081] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Signalling peer 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:32.073729 27218 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:57654
I20260807 01:47:32.074383 27218 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:32.074754 27218 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:32.080363 27105 raft_consensus.cc:2955] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Committing config change with OpId 6.6: config changed from index -1 to 6, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:32.086880 26948 catalog_manager.cc:5697] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index -1 to 6, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 6 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:32.098558 27218 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:32.106942 27218 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:32.110140 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:32.110134 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:32.110967 27333 raft_consensus.cc:3055] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Stepping down as leader of term 6
I20260807 01:47:32.111121 28715 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:32.111354 27333 raft_consensus.cc:740] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:32.112183 27333 consensus_queue.cc:260] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:32.113590 27333 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:32.119827 28715 raft_consensus.cc:2468] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:47:32.123996 27175 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:32.124995 30146 raft_consensus.cc:2804] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:47:32.125562 30146 raft_consensus.cc:697] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 7 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:32.126436 30146 consensus_queue.cc:237] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:32.121482 27105 consensus_queue.cc:237] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:32.133647 27333 raft_consensus.cc:2468] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:47:32.158063 30314 raft_consensus.cc:993] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade: : Instructing follower 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:32.158639 30314 raft_consensus.cc:1081] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Signalling peer 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:32.160575 27218 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:57654
I20260807 01:47:32.161222 27218 raft_consensus.cc:493] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:32.161581 27218 raft_consensus.cc:3060] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:32.183990 27337 raft_consensus.cc:1275] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260807 01:47:32.188889 27067 consensus_peers.cc:597] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 54eb75662df043488517f09bff1dfcff. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:32.190601 30171 consensus_queue.cc:1048] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:32.206902 27227 raft_consensus.cc:1275] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:47:32.209499 30087 consensus_queue.cc:1048] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:32.218863 27218 raft_consensus.cc:515] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:32.231038 27218 leader_election.cc:290] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:32.232555 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:32.231714 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:32.233126 27333 raft_consensus.cc:3055] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:32.233458 27333 raft_consensus.cc:740] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:32.234278 27333 consensus_queue.cc:260] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:32.235713 27333 raft_consensus.cc:3060] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:32.235827 29987 raft_consensus.cc:2955] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:32.237612 27216 raft_consensus.cc:2955] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:32.242972 30349 raft_consensus.cc:993] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:32.243637 30349 raft_consensus.cc:1081] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:32.246248 27105 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a0e280c976844349b4874a479aa08de2"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:34468
I20260807 01:47:32.253811 27105 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:32.257545 27105 raft_consensus.cc:3060] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:32.233479 28715 raft_consensus.cc:3060] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:32.258052 27337 raft_consensus.cc:2955] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:32.302487 27105 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:32.308888 28715 raft_consensus.cc:2468] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:47:32.288116 27333 raft_consensus.cc:2468] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:47:32.310920 27170 leader_election.cc:304] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:32.311599 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:32.312098 27105 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:32.312493 27337 raft_consensus.cc:3055] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:32.313103 27337 raft_consensus.cc:740] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:32.314333 27337 consensus_queue.cc:260] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:32.314795 30146 raft_consensus.cc:2804] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:47:32.315549 30146 raft_consensus.cc:697] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:32.316779 30146 consensus_queue.cc:237] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:32.328631 26950 catalog_manager.cc:5697] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index -1 to 4, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 4 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:32.335600 27337 raft_consensus.cc:3060] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:32.312494 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:32.341017 28715 raft_consensus.cc:3060] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:32.344513 27337 raft_consensus.cc:2468] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
W20260807 01:47:32.345224 27036 auto_rebalancer.cc:644] Failed to schedule move for tablet 1c4e145f8f52431fa41b1f596e2f1133: Illegal state: Leader has not yet committed an operation in its own term
I20260807 01:47:32.346238 27063 leader_election.cc:304] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:32.347154 29987 raft_consensus.cc:2804] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:47:32.347656 29987 raft_consensus.cc:697] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:32.348577 29987 consensus_queue.cc:237] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:32.358312 28715 raft_consensus.cc:2468] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:32.362195 27105 consensus_queue.cc:237] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:32.374413 26948 catalog_manager.cc:5697] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 6 to 7, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 7 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:32.398547 26950 catalog_manager.cc:5697] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 1 to 2, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 2 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260807 01:47:32.399747 27067 consensus_peers.cc:597] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b4fd523ba5d463b8508a943e293612c. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:32.401736 27216 raft_consensus.cc:1275] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 01:47:32.406219 29987 consensus_queue.cc:1048] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:32.417142 27333 raft_consensus.cc:1275] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260807 01:47:32.421504 30171 consensus_queue.cc:1048] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:32.432448 26950 catalog_manager.cc:5697] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 2, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 2 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:32.465400 27218 raft_consensus.cc:1275] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 01:47:32.467536 30087 consensus_queue.cc:1048] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260807 01:47:32.518060 30387 ts_tablet_manager.cc:933] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:47:32.536036 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 77771.1us
I20260807 01:47:32.545076 30171 raft_consensus.cc:2955] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
W20260807 01:47:32.551949 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 122456us
W20260807 01:47:32.560503 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 59378.3us
I20260807 01:47:32.563805 27330 raft_consensus.cc:1275] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 01:47:32.564273 27218 raft_consensus.cc:2955] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:32.575191 30356 consensus_queue.cc:1048] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:32.583488 30387 tablet_copy_client.cc:323] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.3:42429
I20260807 01:47:32.587608 27330 raft_consensus.cc:2955] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:32.591516 27350 tablet_copy_service.cc:140] P fcf4e45defb94276bcfa68392711dade: Received BeginTabletCopySession request for tablet aff03a52fbfd498ab6d04a579e96fd0e from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:36604)
I20260807 01:47:32.592674 27350 tablet_copy_service.cc:161] P fcf4e45defb94276bcfa68392711dade: Beginning new tablet copy session on tablet aff03a52fbfd498ab6d04a579e96fd0e from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:36604: session id = 57bb4985157447f0a3ab680fa0026f12-aff03a52fbfd498ab6d04a579e96fd0e
I20260807 01:47:32.596695 26950 catalog_manager.cc:5697] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index -1 to 3, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 2 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:32.612113 27333 raft_consensus.cc:1275] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260807 01:47:32.629787 30392 consensus_queue.cc:1048] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260807 01:47:32.634754 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 74544us
I20260807 01:47:32.648257 27350 tablet_copy_source_session.cc:215] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:32.836308 30387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aff03a52fbfd498ab6d04a579e96fd0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:32.879187 30411 ts_tablet_manager.cc:933] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
W20260807 01:47:32.900861 30401 log.cc:927] Time spent T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.165s user 0.000s sys 0.006s
W20260807 01:47:32.931481 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 252590us
I20260807 01:47:32.942508 28715 raft_consensus.cc:1275] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260807 01:47:32.947701 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:32.952186 28715 raft_consensus.cc:1275] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260807 01:47:32.966410 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 179324us
W20260807 01:47:33.017275 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 51196.2us
I20260807 01:47:32.989681 28715 raft_consensus.cc:1275] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:33.033123 30411 tablet_copy_client.cc:323] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.1:35499
W20260807 01:47:33.131562 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 88110.4us
I20260807 01:47:32.976699 30394 consensus_queue.cc:1048] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.134s
I20260807 01:47:33.058585 30419 consensus_queue.cc:1048] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
I20260807 01:47:33.209565 27330 raft_consensus.cc:1275] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:33.224133 27127 tablet_copy_service.cc:140] P 3262a9d183914b699227fdba0ca5bed3: Received BeginTabletCopySession request for tablet 54eb75662df043488517f09bff1dfcff from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:52556)
I20260807 01:47:33.224799 27127 tablet_copy_service.cc:161] P 3262a9d183914b699227fdba0ca5bed3: Beginning new tablet copy session on tablet 54eb75662df043488517f09bff1dfcff from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:52556: session id = 57bb4985157447f0a3ab680fa0026f12-54eb75662df043488517f09bff1dfcff
I20260807 01:47:33.212069 30421 consensus_queue.cc:1048] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260807 01:47:33.262876 30435 ts_tablet_manager.cc:933] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
W20260807 01:47:33.277262 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 46330us
I20260807 01:47:33.299317 30421 consensus_queue.cc:1048] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
W20260807 01:47:33.326821 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 300263us
W20260807 01:47:33.368114 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 106701us
W20260807 01:47:33.405917 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 112502us
W20260807 01:47:33.443883 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 78878.1us
I20260807 01:47:33.483788 27216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:33.484863 27216 raft_consensus.cc:606] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:33.489018 27216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:33.489789 27216 raft_consensus.cc:606] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:33.496021 27110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:33.496913 27110 raft_consensus.cc:606] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:33.503582 30387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:48362 (request call id 527) took 1121 ms. Trace:
I20260807 01:47:33.504303 30387 rpcz_store.cc:276] 0807 01:47:32.382425 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:32.387280 (+ 4855us) service_pool.cc:224] Handling call
0807 01:47:32.518631 (+131351us) ts_tablet_manager.cc:934] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
0807 01:47:32.691503 (+172872us) tablet_copy_client.cc:368] Tablet copy session begun
0807 01:47:33.225402 (+533899us) tablet_metadata.cc:665] Metadata flushed
0807 01:47:33.225544 (+ 142us) tablet_copy_client.cc:497] Wrote new tablet metadata
0807 01:47:33.325695 (+100151us) tablet_copy_client.cc:418] Wrote new consensus metadata
0807 01:47:33.326077 (+ 382us) tablet_replica.cc:206] Creating consensus instance
0807 01:47:33.503362 (+177285us) ts_tablet_manager.cc:989] Replica registered.
0807 01:47:33.503470 (+ 108us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":432,"tablet-copy.queue_time_us":130131,"thread_start_us":133929,"threads_started":2}
I20260807 01:47:33.551560 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260807 01:47:33.538142 30435 tablet_copy_client.cc:323] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.1:35499
I20260807 01:47:33.561873 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:47:33.564410 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
W20260807 01:47:33.576824 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 109456us
I20260807 01:47:33.584996 27127 tablet_copy_source_session.cc:215] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:33.569774 30387 tablet_copy_client.cc:806] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:33.610818 30387 tablet_copy_client.cc:670] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
W20260807 01:47:33.594595 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 144400us
I20260807 01:47:33.664059 30411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54eb75662df043488517f09bff1dfcff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:33.667932 27126 tablet_copy_service.cc:140] P 3262a9d183914b699227fdba0ca5bed3: Received BeginTabletCopySession request for tablet 2b4fd523ba5d463b8508a943e293612c from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:52556)
I20260807 01:47:33.669152 27126 tablet_copy_service.cc:161] P 3262a9d183914b699227fdba0ca5bed3: Beginning new tablet copy session on tablet 2b4fd523ba5d463b8508a943e293612c from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:52556: session id = 57bb4985157447f0a3ab680fa0026f12-2b4fd523ba5d463b8508a943e293612c
W20260807 01:47:33.707559 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 77954.9us
W20260807 01:47:33.710527 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 265321us
W20260807 01:47:33.714982 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 69731.1us
I20260807 01:47:33.780316 30387 tablet_copy_client.cc:538] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260807 01:47:33.978580 30474 log.cc:927] Time spent T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.154s user 0.005s sys 0.000s
I20260807 01:47:33.993561 27126 tablet_copy_source_session.cc:215] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3: Tablet Copy: opened 0 blocks and 1 log segments
W20260807 01:47:34.017037 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 44130.6us
I20260807 01:47:34.050784 30429 raft_consensus.cc:493] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:34.051433 30429 raft_consensus.cc:515] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:34.067277 30435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b4fd523ba5d463b8508a943e293612c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:34.049083 30315 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:34.101310 30387 tablet_bootstrap.cc:492] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:34.116176 30448 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:34.111058 30161 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:34.151351 30448 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:34.153671 30448 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:34.097045 30315 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:34.180657 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 125676us
I20260807 01:47:34.161557 30161 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:34.214519 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 57253us
I20260807 01:47:34.216696 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:34.218178 30482 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:34.219211 30482 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:34.189353 30429 leader_election.cc:290] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:34.224359 30417 raft_consensus.cc:993] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:34.232424 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:34.229984 30161 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:34.242689 30412 raft_consensus.cc:1081] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:34.251746 27099 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:41080
W20260807 01:47:34.252431 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 48867.3us
I20260807 01:47:34.267652 27337 raft_consensus.cc:1275] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:34.273118 30315 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:34.265869 27099 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:34.274297 27099 raft_consensus.cc:3060] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:34.250144 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:34.235095 30482 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:34.330071 30466 raft_consensus.cc:993] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:34.331214 30466 raft_consensus.cc:1081] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:34.335336 27099 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:34.339011 30314 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:34.340811 27099 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:34.340327 30314 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:34.346517 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:34.347230 27218 raft_consensus.cc:3055] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:34.347563 27218 raft_consensus.cc:740] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:34.351436 27100 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:41080
I20260807 01:47:34.352087 27100 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:34.352445 27100 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:34.361320 27218 consensus_queue.cc:260] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:34.371737 30314 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:34.389123 30395 raft_consensus.cc:993] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:34.389959 30395 raft_consensus.cc:1081] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Signalling peer fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:34.398291 27109 raft_consensus.cc:1240] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 4. Current term is 5. Ops: []
I20260807 01:47:34.403352 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:34.404234 27218 raft_consensus.cc:3060] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:34.410362 30411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:52322 (request call id 386) took 1657 ms. Trace:
I20260807 01:47:34.410995 30411 rpcz_store.cc:276] 0807 01:47:32.752809 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:32.753229 (+ 420us) service_pool.cc:224] Handling call
0807 01:47:32.879874 (+126645us) ts_tablet_manager.cc:934] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
0807 01:47:33.661871 (+781997us) tablet_copy_client.cc:368] Tablet copy session begun
0807 01:47:33.975279 (+313408us) tablet_metadata.cc:665] Metadata flushed
0807 01:47:33.975515 (+ 236us) tablet_copy_client.cc:497] Wrote new tablet metadata
0807 01:47:34.315407 (+339892us) tablet_copy_client.cc:418] Wrote new consensus metadata
0807 01:47:34.348723 (+ 33316us) tablet_replica.cc:206] Creating consensus instance
0807 01:47:34.398822 (+ 50099us) ts_tablet_manager.cc:989] Replica registered.
0807 01:47:34.410263 (+ 11441us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":291,"tablet-copy.queue_time_us":124954,"thread_start_us":188512,"threads_started":2}
I20260807 01:47:34.412853 30411 tablet_copy_client.cc:806] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:34.413702 30411 tablet_copy_client.cc:670] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:34.426728 30411 tablet_copy_client.cc:538] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:34.387432 27100 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:34.449507 27109 raft_consensus.cc:1240] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 4. Current term is 5. Ops: []
I20260807 01:47:34.449003 30411 tablet_bootstrap.cc:492] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:34.460178 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:34.464188 30417 consensus_queue.cc:1059] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:34.466084 30466 raft_consensus.cc:3055] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:34.466560 30466 raft_consensus.cc:740] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:34.477051 27218 raft_consensus.cc:2468] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:34.478720 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:34.478569 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:34.457710 27100 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:34.479617 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:34.482270 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:34.484328 30466 consensus_queue.cc:260] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:34.485093 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:34.486996 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:34.496779 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:34.497363 27064 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:34.503716 30146 consensus_queue.cc:1048] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:34.506245 30466 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:34.507839 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:34.516645 29987 raft_consensus.cc:2804] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:47:34.518908 29987 raft_consensus.cc:697] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:34.528194 29987 consensus_queue.cc:237] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:34.542652 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:34.543828 27328 raft_consensus.cc:3060] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:34.555145 27327 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:33976
I20260807 01:47:34.556586 27327 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:34.557377 27327 raft_consensus.cc:3060] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:34.560377 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:34.561072 27337 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:34.573387 27337 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:34.591957 27327 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:34.602699 27330 raft_consensus.cc:1240] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 1. Current term is 2. Ops: []
I20260807 01:47:34.611444 27327 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:34.668912 30435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:52322 (request call id 389) took 1507 ms. Trace:
I20260807 01:47:34.669394 30435 rpcz_store.cc:276] 0807 01:47:33.161983 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:33.162198 (+ 215us) service_pool.cc:224] Handling call
0807 01:47:33.263374 (+101176us) ts_tablet_manager.cc:934] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
0807 01:47:34.055683 (+792309us) tablet_copy_client.cc:368] Tablet copy session begun
0807 01:47:34.429482 (+373799us) tablet_metadata.cc:665] Metadata flushed
0807 01:47:34.429618 (+ 136us) tablet_copy_client.cc:497] Wrote new tablet metadata
0807 01:47:34.668033 (+238415us) tablet_copy_client.cc:418] Wrote new consensus metadata
0807 01:47:34.668338 (+ 305us) tablet_replica.cc:206] Creating consensus instance
0807 01:47:34.668787 (+ 449us) ts_tablet_manager.cc:989] Replica registered.
0807 01:47:34.668845 (+ 58us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":96799,"tablet-copy.queue_time_us":100042,"thread_start_us":250176,"threads_started":2}
I20260807 01:47:34.670814 30435 tablet_copy_client.cc:806] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:34.671380 30435 tablet_copy_client.cc:670] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:34.672199 26950 catalog_manager.cc:5697] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 4 to 5, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 5 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
W20260807 01:47:34.674628 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 83616.1us
I20260807 01:47:34.683607 27328 raft_consensus.cc:2468] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:34.699756 27287 leader_election.cc:304] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:34.716035 30484 raft_consensus.cc:2749] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:34.723899 27287 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:34.745911 30488 raft_consensus.cc:2749] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:47:34.746393 30387 tablet_bootstrap.cc:492] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:34.747732 27287 leader_election.cc:304] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:34.748793 27287 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:34.751190 30484 raft_consensus.cc:2749] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:47:34.751255 30488 raft_consensus.cc:2749] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:47:34.751181 30435 tablet_copy_client.cc:538] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:34.757812 30387 tablet_bootstrap.cc:492] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:34.749981 27230 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:34.765364 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:34.766269 30387 ts_tablet_manager.cc:1403] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.665s user 0.044s sys 0.029s
I20260807 01:47:34.766286 27100 raft_consensus.cc:3055] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:34.767158 27100 raft_consensus.cc:740] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:34.767864 27100 consensus_queue.cc:260] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:34.769270 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:34.770032 27227 raft_consensus.cc:3060] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:34.769773 30387 raft_consensus.cc:359] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 6 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:34.770676 30387 raft_consensus.cc:740] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 6 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:34.770833 30406 consensus_queue.cc:1059] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:47:34.771385 30387 consensus_queue.cc:260] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:34.773864 30387 ts_tablet_manager.cc:1434] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260807 01:47:34.784327 27350 tablet_copy_service.cc:342] P fcf4e45defb94276bcfa68392711dade: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-aff03a52fbfd498ab6d04a579e96fd0e received from {username='slave'} at 127.0.0.1:36604
I20260807 01:47:34.785573 27350 tablet_copy_service.cc:434] P fcf4e45defb94276bcfa68392711dade: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-aff03a52fbfd498ab6d04a579e96fd0e on tablet aff03a52fbfd498ab6d04a579e96fd0e with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:34.786407 27287 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:34.787421 30487 raft_consensus.cc:2749] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:47:34.788476 27287 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:34.789284 30488 raft_consensus.cc:2749] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:34.795341 27227 raft_consensus.cc:2468] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:47:34.797350 30411 tablet_bootstrap.cc:492] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:34.799307 27064 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:34.800097 30446 raft_consensus.cc:2804] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:47:34.800745 30446 raft_consensus.cc:697] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:34.801885 30446 consensus_queue.cc:237] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:34.813589 30411 tablet_bootstrap.cc:492] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:34.816203 27100 raft_consensus.cc:3060] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:34.831794 27100 raft_consensus.cc:2468] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
W20260807 01:47:34.814689 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 46471.6us
I20260807 01:47:34.833817 30435 tablet_bootstrap.cc:492] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:34.838991 30411 ts_tablet_manager.cc:1403] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.390s user 0.026s sys 0.044s
I20260807 01:47:34.848491 30411 raft_consensus.cc:359] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:34.849094 28709 raft_consensus.cc:1217] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 6 LEARNER]: Deduplicated request from leader. Original: 6.5->[6.6-6.6] Dedup: 6.6->[]
I20260807 01:47:34.849464 30411 raft_consensus.cc:740] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:34.850420 30411 consensus_queue.cc:260] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:34.858142 27287 leader_election.cc:304] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:34.859156 30484 raft_consensus.cc:2804] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:47:34.859687 30484 raft_consensus.cc:697] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:34.860493 30484 consensus_queue.cc:237] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:34.869618 30411 ts_tablet_manager.cc:1434] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.030s user 0.007s sys 0.001s
I20260807 01:47:34.877980 27126 tablet_copy_service.cc:342] P 3262a9d183914b699227fdba0ca5bed3: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-54eb75662df043488517f09bff1dfcff received from {username='slave'} at 127.0.0.1:52556
I20260807 01:47:34.878516 27126 tablet_copy_service.cc:434] P 3262a9d183914b699227fdba0ca5bed3: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-54eb75662df043488517f09bff1dfcff on tablet 54eb75662df043488517f09bff1dfcff with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:34.906350 26954 catalog_manager.cc:5697] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 1 to 2, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 2 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:47:34.921150 30435 tablet_bootstrap.cc:492] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:34.922400 30435 tablet_bootstrap.cc:492] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:34.923174 30435 ts_tablet_manager.cc:1403] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.090s user 0.018s sys 0.018s
I20260807 01:47:34.930217 30435 raft_consensus.cc:359] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:34.931131 30435 raft_consensus.cc:740] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:34.930699 26950 catalog_manager.cc:5697] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 4 to 5, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 5 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:34.931843 30435 consensus_queue.cc:260] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:34.934475 30435 ts_tablet_manager.cc:1434] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.011s user 0.008s sys 0.001s
I20260807 01:47:34.936571 27126 tablet_copy_service.cc:342] P 3262a9d183914b699227fdba0ca5bed3: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-2b4fd523ba5d463b8508a943e293612c received from {username='slave'} at 127.0.0.1:52556
I20260807 01:47:34.937039 27126 tablet_copy_service.cc:434] P 3262a9d183914b699227fdba0ca5bed3: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-2b4fd523ba5d463b8508a943e293612c on tablet 2b4fd523ba5d463b8508a943e293612c with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:35.004067 27227 raft_consensus.cc:1275] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 01:47:35.006049 30406 consensus_queue.cc:1048] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:35.025620 28710 raft_consensus.cc:1217] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20260807 01:47:35.111325 27328 raft_consensus.cc:1275] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 01:47:35.118741 30395 consensus_queue.cc:1048] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:35.258558 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:35.284571 27227 raft_consensus.cc:1275] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:35.285777 30487 raft_consensus.cc:1064] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:35.291625 30506 consensus_queue.cc:1048] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260807 01:47:35.328652 30487 consensus_queue.cc:237] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:35.343272 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 46594.9us
I20260807 01:47:35.364602 28709 raft_consensus.cc:1217] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.3->[4.4-4.4] Dedup: 4.4->[]
I20260807 01:47:35.406011 28715 raft_consensus.cc:1275] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 6 LEARNER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260807 01:47:35.421249 27227 raft_consensus.cc:1275] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260807 01:47:35.438056 30524 consensus_queue.cc:1048] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:47:35.450390 30446 consensus_queue.cc:1048] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
W20260807 01:47:35.511101 30523 log.cc:927] Time spent T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.082s user 0.000s sys 0.006s
I20260807 01:47:35.528326 30501 raft_consensus.cc:1064] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:35.533381 30501 consensus_queue.cc:237] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:35.576565 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 167901us
I20260807 01:47:35.599061 27105 raft_consensus.cc:1275] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 01:47:35.600059 27230 raft_consensus.cc:1275] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260807 01:47:35.600517 27109 raft_consensus.cc:1275] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260807 01:47:35.626226 27227 raft_consensus.cc:1275] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 01:47:35.639971 28709 raft_consensus.cc:1275] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEARNER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 01:47:35.640101 30527 consensus_queue.cc:1048] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 01:47:35.642181 30527 consensus_queue.cc:1048] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 01:47:35.653334 27337 raft_consensus.cc:1275] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260807 01:47:35.664757 30406 consensus_queue.cc:1048] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:35.666647 30446 consensus_queue.cc:1048] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:35.667630 27337 raft_consensus.cc:1275] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260807 01:47:35.682870 30501 consensus_queue.cc:1048] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:47:35.702538 30444 consensus_queue.cc:1048] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:35.714881 30528 consensus_queue.cc:1048] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:47:35.714537 30406 raft_consensus.cc:2955] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:35.740839 27227 raft_consensus.cc:2955] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:35.747620 28710 raft_consensus.cc:2955] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:35.799124 30501 raft_consensus.cc:1064] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:35.802912 30501 consensus_queue.cc:237] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 2, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:35.804060 26949 catalog_manager.cc:5697] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:35.840566 27110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:35.841437 27110 raft_consensus.cc:606] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Received request to transfer leadership to TS 3478729505b046c2a0051c2d0130873f
I20260807 01:47:35.851595 27330 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3806d33df8144ca5a662db5461461b43"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:57092
I20260807 01:47:35.852337 27330 raft_consensus.cc:606] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:35.866854 27110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54eb75662df043488517f09bff1dfcff"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:35.881834 27110 raft_consensus.cc:606] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Received request to transfer leadership to TS 3478729505b046c2a0051c2d0130873f
I20260807 01:47:35.886826 28710 raft_consensus.cc:1275] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 01:47:35.888478 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260807 01:47:35.888219 30487 raft_consensus.cc:2955] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Committing config change with OpId 6.7: config changed from index 6 to 7, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:35.890507 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:47:35.890790 30406 consensus_queue.cc:1048] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:35.905893 27227 raft_consensus.cc:2955] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 6 to 7, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:35.886408 27230 raft_consensus.cc:1275] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 01:47:35.915453 27337 raft_consensus.cc:1275] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 01:47:35.920250 30395 consensus_queue.cc:1048] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:35.920442 27328 raft_consensus.cc:2955] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:35.954694 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:35.957535 30530 raft_consensus.cc:993] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:35.958240 30530 raft_consensus.cc:1081] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:35.913262 27110 raft_consensus.cc:2955] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 6 to 7, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:35.970835 27109 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3806d33df8144ca5a662db5461461b43"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:34468
I20260807 01:47:35.965647 28709 raft_consensus.cc:2955] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 6 to 7, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:35.967443 30406 consensus_queue.cc:1048] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260807 01:47:35.972194 27109 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:35.976042 27109 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:35.985211 30446 raft_consensus.cc:993] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:35.985798 30446 raft_consensus.cc:1081] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Signalling peer 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:35.987558 27227 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "54eb75662df043488517f09bff1dfcff"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:57662
I20260807 01:47:35.986941 30406 raft_consensus.cc:2955] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:35.988135 27222 raft_consensus.cc:2955] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:36.004168 30395 raft_consensus.cc:993] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:36.004793 30395 raft_consensus.cc:1081] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Signalling peer 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:36.004391 27099 consensus_queue.cc:237] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:36.007081 27230 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:57662
I20260807 01:47:36.007654 27230 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:36.007984 27230 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:36.013506 28710 raft_consensus.cc:1275] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:36.016327 30395 consensus_queue.cc:1048] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:36.018313 27216 raft_consensus.cc:1275] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:36.023080 30395 consensus_queue.cc:1048] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:47:36.029253 26948 catalog_manager.cc:5697] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index 6 to 7, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 6 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:36.033900 27109 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:36.049154 27109 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:36.038609 28710 raft_consensus.cc:2955] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:36.053072 28709 raft_consensus.cc:2955] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:36.062122 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:36.062879 27214 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:36.083837 27330 consensus_queue.cc:237] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:36.035552 30446 raft_consensus.cc:2955] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:36.089368 27227 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:36.089977 27227 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:36.120790 27214 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:36.051503 27230 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:36.131455 27214 raft_consensus.cc:2955] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:36.132473 27064 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:36.157387 28715 raft_consensus.cc:1275] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260807 01:47:36.157872 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:36.172847 27328 raft_consensus.cc:3055] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:36.173254 27328 raft_consensus.cc:740] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:36.174538 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:36.176249 27109 raft_consensus.cc:3055] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:36.176720 27109 raft_consensus.cc:740] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:36.175601 30526 consensus_queue.cc:1048] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260807 01:47:36.177785 27109 consensus_queue.cc:260] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:36.180620 27109 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:36.182029 27216 raft_consensus.cc:1275] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260807 01:47:36.182536 27328 consensus_queue.cc:260] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:36.187528 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:36.188201 27327 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:36.189441 30484 raft_consensus.cc:2955] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Committing config change with OpId 6.8: config changed from index 7 to 8, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:36.193362 30487 consensus_queue.cc:1048] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 01:47:36.199035 27327 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:36.167902 27230 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:36.192771 28715 raft_consensus.cc:2955] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Committing config change with OpId 6.8: config changed from index 7 to 8, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:36.175631 30444 raft_consensus.cc:2804] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:47:36.206614 30444 raft_consensus.cc:697] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:36.207597 30444 consensus_queue.cc:237] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:36.183984 27328 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:36.210131 27227 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:36.217059 27227 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:36.217343 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:36.217154 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:36.218289 27099 raft_consensus.cc:3055] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:36.218611 27099 raft_consensus.cc:740] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:36.219277 27099 consensus_queue.cc:260] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:36.220343 27099 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
W20260807 01:47:36.222622 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 54766.2us
I20260807 01:47:36.234027 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2b4fd523ba5d463b8508a943e293612c with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:36.236572 27170 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:36.238140 30466 raft_consensus.cc:2804] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:36.238770 30466 raft_consensus.cc:697] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:36.239694 30466 consensus_queue.cc:237] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:36.239339 27222 raft_consensus.cc:2955] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Committing config change with OpId 6.8: config changed from index 7 to 8, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:36.247732 26954 catalog_manager.cc:5697] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 4 to 5, VOTER fcf4e45defb94276bcfa68392711dade (127.24.4.3) evicted. New cstate: current_term: 2 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:36.260099 27099 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:47:36.262688 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:36.270867 27317 tablet_service.cc:1558] Processing DeleteTablet for tablet 2b4fd523ba5d463b8508a943e293612c with delete_type TABLET_DATA_TOMBSTONED (TS fcf4e45defb94276bcfa68392711dade not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:59474
I20260807 01:47:36.274926 30554 tablet_replica.cc:333] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade: stopping tablet replica
I20260807 01:47:36.262602 27327 raft_consensus.cc:2955] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:36.276409 30554 raft_consensus.cc:2243] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:36.277575 30554 raft_consensus.cc:2272] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:36.282809 30554 ts_tablet_manager.cc:1916] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:36.292017 26931 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet aff03a52fbfd498ab6d04a579e96fd0e with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:36.295158 27328 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:36.159941 26950 catalog_manager.cc:5697] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index 4 to 5, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:36.302047 27170 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:36.303386 27109 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:36.304420 30466 raft_consensus.cc:2749] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 01:47:36.309286 30554 ts_tablet_manager.cc:1929] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260807 01:47:36.314735 30554 log.cc:1199] T 2b4fd523ba5d463b8508a943e293612c P fcf4e45defb94276bcfa68392711dade: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-2-root/wals/2b4fd523ba5d463b8508a943e293612c
I20260807 01:47:36.316982 26931 catalog_manager.cc:5028] TS fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): tablet 2b4fd523ba5d463b8508a943e293612c (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:36.330555 26949 catalog_manager.cc:5697] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index 7 to 8, VOTER 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) evicted. New cstate: current_term: 6 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:36.357359 27088 tablet_service.cc:1558] Processing DeleteTablet for tablet aff03a52fbfd498ab6d04a579e96fd0e with delete_type TABLET_DATA_TOMBSTONED (TS 3262a9d183914b699227fdba0ca5bed3 not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:41078
I20260807 01:47:36.362844 26948 catalog_manager.cc:5697] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 4 to 5, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 5 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:36.372445 30559 tablet_replica.cc:333] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3: stopping tablet replica
I20260807 01:47:36.374080 30559 raft_consensus.cc:2243] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:36.375145 30559 raft_consensus.cc:2272] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:36.380650 30559 ts_tablet_manager.cc:1916] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:36.384931 26952 catalog_manager.cc:5697] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 3 to 4, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 4 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:36.409425 30559 ts_tablet_manager.cc:1929] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 6.7
I20260807 01:47:36.410243 30559 log.cc:1199] T aff03a52fbfd498ab6d04a579e96fd0e P 3262a9d183914b699227fdba0ca5bed3: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-0-root/wals/aff03a52fbfd498ab6d04a579e96fd0e
I20260807 01:47:36.412743 26932 catalog_manager.cc:5028] TS 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): tablet aff03a52fbfd498ab6d04a579e96fd0e (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:36.632404 27109 raft_consensus.cc:1275] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:47:36.633900 30441 consensus_queue.cc:1048] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:36.676019 27230 raft_consensus.cc:1275] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 01:47:36.678583 30444 consensus_queue.cc:1048] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:36.695297 27330 raft_consensus.cc:1275] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:47:36.699626 30466 consensus_queue.cc:1048] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:36.718215 27328 raft_consensus.cc:1275] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 01:47:36.720603 30395 consensus_queue.cc:1048] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:37.366219 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:37.423302 27109 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3806d33df8144ca5a662db5461461b43"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:37.423964 27109 raft_consensus.cc:606] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:37.428967 27214 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:37.429694 27214 raft_consensus.cc:606] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:37.433822 27109 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:37.434485 27109 raft_consensus.cc:606] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Received request to transfer leadership to TS 3478729505b046c2a0051c2d0130873f
I20260807 01:47:37.437242 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260807 01:47:37.438916 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:47:37.439775 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:37.510950 30571 raft_consensus.cc:993] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:37.511559 30571 raft_consensus.cc:1081] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Signalling peer 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:37.513197 27214 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:57662
I20260807 01:47:37.513687 27214 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:37.513969 27214 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:37.520859 27214 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:37.523849 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:37.524682 27109 raft_consensus.cc:3055] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:37.525007 27109 raft_consensus.cc:740] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:37.526587 27109 consensus_queue.cc:260] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:37.527743 27109 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:37.523849 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:37.530467 27328 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:37.537783 27109 raft_consensus.cc:2468] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:47:37.539201 27172 leader_election.cc:304] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:37.539482 27214 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:37.541618 30466 raft_consensus.cc:2804] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:47:37.542171 30466 raft_consensus.cc:697] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:37.543571 30466 consensus_queue.cc:237] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:37.548599 27328 raft_consensus.cc:2468] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:47:37.555058 30570 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:37.555672 30570 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:37.561506 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:37.561056 26952 catalog_manager.cc:5697] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 2 to 3, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 3 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:37.563400 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:37.565337 27214 raft_consensus.cc:2393] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 5: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:37.562318 27109 raft_consensus.cc:2393] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 5: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:37.568250 28660 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:37.569306 30485 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:37.574981 30570 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:37.576277 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:37.580195 30485 raft_consensus.cc:2749] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:37.639860 30574 raft_consensus.cc:993] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:37.640478 30466 raft_consensus.cc:1081] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:37.642035 27109 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:41080
I20260807 01:47:37.642596 27109 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:37.642884 27109 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:37.649048 27109 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:37.650661 27109 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:37.651448 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:37.651948 27230 raft_consensus.cc:3055] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:37.652171 27230 raft_consensus.cc:740] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:37.652797 27230 consensus_queue.cc:260] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:37.653599 27230 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:37.656951 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:37.657585 27328 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:37.661736 27230 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:47:37.663283 27064 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:37.664608 30444 raft_consensus.cc:2804] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:37.665126 30444 raft_consensus.cc:697] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:37.666095 30444 consensus_queue.cc:237] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:37.666988 27328 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:47:37.681238 26952 catalog_manager.cc:5697] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 3 to 4, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 4 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:37.744771 30444 raft_consensus.cc:993] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:37.745347 30444 raft_consensus.cc:1081] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Signalling peer fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:47:37.747077 27327 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3806d33df8144ca5a662db5461461b43"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:33976
I20260807 01:47:37.747659 27327 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:37.747993 27327 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:37.755401 27327 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:37.757454 27327 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:37.758507 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:37.758701 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:37.759176 27099 raft_consensus.cc:3055] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:47:37.759186 27230 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:37.759600 27099 raft_consensus.cc:740] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:37.760418 27099 consensus_queue.cc:260] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:37.761641 27099 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:37.763520 30314 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:37.767395 27230 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:47:37.768800 27287 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:37.769693 30487 raft_consensus.cc:2804] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:47:37.769893 30314 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:37.774797 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:37.774817 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:37.775647 28710 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:47:37.775815 27214 raft_consensus.cc:2393] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 5: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:37.777590 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:37.778501 27109 raft_consensus.cc:2393] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 5: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:37.779194 30314 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:37.779544 27287 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:37.780896 30314 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:37.794531 30314 raft_consensus.cc:2749] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:37.794869 30487 raft_consensus.cc:697] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:37.795918 30487 consensus_queue.cc:237] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:37.799587 27099 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:47:37.814613 26952 catalog_manager.cc:5697] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 5 to 6, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 6 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:47:37.943930 30444 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:37.944494 30444 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:37.947638 30444 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:37.947793 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:37.948051 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:37.948433 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:37.948779 27214 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:37.948829 28710 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:37.949044 27328 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:37.950444 27064 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:37.951275 30444 raft_consensus.cc:2804] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:47:37.951738 30444 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:37.952003 30444 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:37.959753 30444 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:37.962173 30444 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:37.963325 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:37.963960 27328 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:37.976581 27328 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:47:37.977257 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:37.977988 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:37.978608 28710 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:37.980710 27214 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:37.985252 28710 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:47:37.986742 27067 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:37.987699 30571 raft_consensus.cc:2804] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:47:37.988245 30571 raft_consensus.cc:697] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:37.988782 27214 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:47:37.989181 30571 consensus_queue.cc:237] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:38.002852 26952 catalog_manager.cc:5697] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:38.087074 27327 raft_consensus.cc:1275] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:47:38.088644 30466 consensus_queue.cc:1048] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:38.136204 27099 raft_consensus.cc:1275] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:47:38.138864 30592 consensus_queue.cc:1048] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:38.178963 27109 raft_consensus.cc:1275] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260807 01:47:38.180680 30584 consensus_queue.cc:1048] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:47:38.203255 27214 raft_consensus.cc:1275] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260807 01:47:38.209336 30314 consensus_queue.cc:1048] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:47:38.243079 27327 raft_consensus.cc:1275] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:47:38.244519 30444 consensus_queue.cc:1048] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:38.262844 27214 raft_consensus.cc:1275] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:47:38.265115 30444 consensus_queue.cc:1048] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:38.371682 27327 raft_consensus.cc:1275] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260807 01:47:38.373140 30608 consensus_queue.cc:1048] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 01:47:38.496083 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:38.568596 27230 raft_consensus.cc:1275] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260807 01:47:38.568979 28715 raft_consensus.cc:1275] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260807 01:47:38.571691 30571 consensus_queue.cc:1048] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:47:38.585354 30608 consensus_queue.cc:1048] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:47:38.612115 27109 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:38.613013 27109 raft_consensus.cc:606] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Received request to transfer leadership to TS 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:38.616525 27328 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3806d33df8144ca5a662db5461461b43"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
mode: GRACEFUL
new_leader_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:57092
I20260807 01:47:38.617165 27328 raft_consensus.cc:606] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Received request to transfer leadership to TS 3478729505b046c2a0051c2d0130873f
I20260807 01:47:38.628036 27214 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:38.628716 27214 raft_consensus.cc:606] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:38.633375 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260807 01:47:38.639196 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:47:38.640064 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
W20260807 01:47:38.640275 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 60293.7us
W20260807 01:47:38.658267 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 67700.6us
I20260807 01:47:38.667399 30608 raft_consensus.cc:993] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:47:38.668054 30608 raft_consensus.cc:1081] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Signalling peer 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:47:38.672096 28715 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:52322
I20260807 01:47:38.672703 28715 raft_consensus.cc:493] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:38.673102 28715 raft_consensus.cc:3060] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:38.698577 28715 raft_consensus.cc:515] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:38.702904 28715 leader_election.cc:290] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:38.711863 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:38.712544 27099 raft_consensus.cc:3055] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:38.712848 27099 raft_consensus.cc:740] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:38.713632 27099 consensus_queue.cc:260] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:38.714375 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:38.714999 27327 raft_consensus.cc:3060] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:38.715201 27099 raft_consensus.cc:3060] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:38.728403 27099 raft_consensus.cc:2468] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:47:38.730104 28660 leader_election.cc:304] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:38.731199 30599 raft_consensus.cc:2804] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:47:38.732019 30599 raft_consensus.cc:697] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:38.733057 30599 consensus_queue.cc:237] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:38.743579 27327 raft_consensus.cc:2468] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:47:38.743644 27099 consensus_queue.cc:237] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:38.751677 27327 raft_consensus.cc:1275] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260807 01:47:38.753239 30444 consensus_queue.cc:1048] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 01:47:38.755404 28715 raft_consensus.cc:1275] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260807 01:47:38.757529 30444 consensus_queue.cc:1048] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 01:47:38.761469 30607 raft_consensus.cc:2955] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Committing config change with OpId 6.7: config changed from index 5 to 7, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:38.763314 26954 catalog_manager.cc:5697] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 2 to 3, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4). New cstate: current_term: 3 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:38.772032 27328 raft_consensus.cc:2955] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 5 to 7, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:38.778918 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 54eb75662df043488517f09bff1dfcff with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:38.782500 28715 raft_consensus.cc:2955] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 5 to 7, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:38.785143 26952 catalog_manager.cc:5697] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index 5 to 7, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New cstate: current_term: 6 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:38.813393 27199 tablet_service.cc:1558] Processing DeleteTablet for tablet 54eb75662df043488517f09bff1dfcff with delete_type TABLET_DATA_TOMBSTONED (TS 3478729505b046c2a0051c2d0130873f not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:53780
I20260807 01:47:38.816293 30624 tablet_replica.cc:333] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f: stopping tablet replica
I20260807 01:47:38.825270 30314 raft_consensus.cc:993] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade: : Instructing follower 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:38.825892 30314 raft_consensus.cc:1081] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Signalling peer 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:38.828465 27222 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3806d33df8144ca5a662db5461461b43"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:57654
I20260807 01:47:38.829069 27222 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:38.829411 27222 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:38.817869 30624 raft_consensus.cc:2243] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:38.836099 30624 raft_consensus.cc:2272] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:38.841063 30624 ts_tablet_manager.cc:1916] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:38.841889 27222 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:38.844555 27222 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:38.845920 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:38.845824 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:38.847160 27327 raft_consensus.cc:3055] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Stepping down as leader of term 6
I20260807 01:47:38.847501 27327 raft_consensus.cc:740] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:38.846602 27109 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:38.848275 27327 consensus_queue.cc:260] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:38.851927 27327 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:38.861603 27109 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:47:38.862298 30624 ts_tablet_manager.cc:1929] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 6.6
I20260807 01:47:38.862782 30624 log.cc:1199] T 54eb75662df043488517f09bff1dfcff P 3478729505b046c2a0051c2d0130873f: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-1-root/wals/54eb75662df043488517f09bff1dfcff
I20260807 01:47:38.862845 27172 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:38.863534 27327 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:47:38.863723 30574 raft_consensus.cc:2804] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:47:38.864650 30574 raft_consensus.cc:697] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 7 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:38.865114 26932 catalog_manager.cc:5028] TS 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): tablet 54eb75662df043488517f09bff1dfcff (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
I20260807 01:47:38.865957 30574 consensus_queue.cc:237] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:38.876935 26948 catalog_manager.cc:5697] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 6 to 7, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 7 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:38.985244 30592 raft_consensus.cc:993] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:38.985703 30574 raft_consensus.cc:1081] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:38.987325 27109 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:41080
I20260807 01:47:38.987787 27109 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:38.988030 27109 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:38.992985 27109 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:38.994637 27109 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:38.995460 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:38.995697 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:38.996107 27230 raft_consensus.cc:3055] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:38.996438 27230 raft_consensus.cc:740] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:38.996393 28715 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:38.997426 27230 consensus_queue.cc:260] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:38.999428 27230 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:39.003372 28715 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:39.004654 27067 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:39.005535 30607 raft_consensus.cc:2804] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:47:39.006027 30607 raft_consensus.cc:697] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:39.007007 27230 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:39.006942 30607 consensus_queue.cc:237] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:39.020320 26948 catalog_manager.cc:5697] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 4 to 5, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 5 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:39.198138 27099 raft_consensus.cc:1275] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260807 01:47:39.199417 30599 consensus_queue.cc:1048] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 01:47:39.221683 27330 raft_consensus.cc:1275] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260807 01:47:39.229231 30632 consensus_queue.cc:1048] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260807 01:47:39.316298 27099 raft_consensus.cc:1275] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260807 01:47:39.317480 30630 consensus_queue.cc:1048] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 01:47:39.324723 27328 raft_consensus.cc:1275] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260807 01:47:39.327003 30574 consensus_queue.cc:1048] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:47:39.512825 28715 raft_consensus.cc:1275] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 01:47:39.514092 30571 consensus_queue.cc:1048] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 01:47:39.525633 27230 raft_consensus.cc:1275] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 01:47:39.527864 30608 consensus_queue.cc:1048] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:47:39.801575 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:39.848798 27214 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:39.849455 27214 raft_consensus.cc:606] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:39.852468 27109 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:39.853086 27109 raft_consensus.cc:606] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Received request to transfer leadership to TS 3478729505b046c2a0051c2d0130873f
I20260807 01:47:39.855034 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260807 01:47:39.857082 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:47:39.858001 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:39.912786 27109 consensus_queue.cc:237] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:39.921511 27328 raft_consensus.cc:1275] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260807 01:47:39.923061 27067 consensus_peers.cc:597] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1c4e145f8f52431fa41b1f596e2f1133. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:39.923404 30571 consensus_queue.cc:1048] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:39.925570 27214 raft_consensus.cc:1275] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260807 01:47:39.927068 30571 consensus_queue.cc:1048] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:39.933081 30645 raft_consensus.cc:2955] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Committing config change with OpId 5.5: config changed from index -1 to 5, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:39.939350 27328 raft_consensus.cc:2955] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index -1 to 5, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:39.949098 27222 consensus_queue.cc:237] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:39.956573 27230 raft_consensus.cc:2955] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index -1 to 5, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:39.957227 26952 catalog_manager.cc:5697] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index -1 to 5, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 5 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260807 01:47:39.967880 27175 consensus_peers.cc:597] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 00395257beb442e892a4ec18c4c30d95. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:39.975881 27328 raft_consensus.cc:1275] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 01:47:39.978039 30574 consensus_queue.cc:1048] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:39.982120 27109 raft_consensus.cc:1275] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 01:47:39.985630 30588 consensus_queue.cc:1048] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:39.998252 30592 raft_consensus.cc:2955] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:40.003316 27328 raft_consensus.cc:2955] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:40.005604 27109 raft_consensus.cc:2955] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:40.034932 27214 consensus_queue.cc:237] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:40.041448 26948 catalog_manager.cc:5697] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade reported cstate change: config changed from index -1 to 5, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 4 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
W20260807 01:47:40.091761 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 65723.4us
W20260807 01:47:40.102101 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 77189.3us
I20260807 01:47:40.106616 27109 raft_consensus.cc:1275] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260807 01:47:40.106974 27175 consensus_peers.cc:597] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: de3b20c9b4a7467ea9e6c098167df7e3. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:40.108083 30588 consensus_queue.cc:1048] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:47:40.109164 27330 raft_consensus.cc:1275] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 01:47:40.110935 30574 consensus_queue.cc:1048] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:47:40.117542 30662 raft_consensus.cc:2955] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:40.129328 30571 raft_consensus.cc:993] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:40.129403 27109 raft_consensus.cc:2955] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:40.130122 30645 raft_consensus.cc:1081] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Signalling peer 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:47:40.129967 27330 raft_consensus.cc:2955] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:40.153954 27214 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:57662
I20260807 01:47:40.154711 27214 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:40.155157 27214 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:40.159394 27100 consensus_queue.cc:237] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:40.165568 26954 catalog_manager.cc:5697] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 1 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:40.172963 27214 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:40.185159 27214 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:40.185390 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:40.186496 28715 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:40.199092 28715 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:47:40.200029 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:40.200794 27109 raft_consensus.cc:3055] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:47:40.201134 27109 raft_consensus.cc:740] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:40.202344 27109 consensus_queue.cc:260] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:40.203759 27109 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:40.208480 27327 raft_consensus.cc:1275] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 01:47:40.210896 27109 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:47:40.211278 27175 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:40.211509 30645 consensus_queue.cc:1048] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:47:40.212438 30662 raft_consensus.cc:2804] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:47:40.213065 30662 raft_consensus.cc:697] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 6 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:40.214058 30662 consensus_queue.cc:237] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:40.217953 27214 raft_consensus.cc:1275] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 01:47:40.223548 30645 consensus_queue.cc:1048] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260807 01:47:40.239423 27067 consensus_peers.cc:597] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 -> Peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225): Couldn't send request to peer 57bb4985157447f0a3ab680fa0026f12. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 67fed460b08e4124a6d4484db52478b1. This is attempt 1: this message will repeat every 5th retry.
I20260807 01:47:40.245736 26952 catalog_manager.cc:5697] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 5 to 6, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 6 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:40.255927 30660 raft_consensus.cc:993] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:40.256476 30660 raft_consensus.cc:1081] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:40.258196 30608 raft_consensus.cc:2955] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:40.262406 27100 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:41080
I20260807 01:47:40.263056 27100 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:40.263437 27100 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:40.268424 27328 raft_consensus.cc:2955] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:40.275226 27100 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:40.278746 27100 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:40.279909 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:40.280718 27214 raft_consensus.cc:3055] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:40.281126 27214 raft_consensus.cc:740] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:40.282433 27214 consensus_queue.cc:260] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:40.285696 27214 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:40.283999 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:40.289204 27327 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:40.289961 26952 catalog_manager.cc:5697] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New cstate: current_term: 1 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:40.298883 27214 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:47:40.300449 27064 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:40.302635 30608 raft_consensus.cc:2804] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:40.303194 30608 raft_consensus.cc:697] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:40.304085 30608 consensus_queue.cc:237] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:40.306659 27222 raft_consensus.cc:2955] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:40.311817 27327 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:47:40.326206 26954 catalog_manager.cc:5697] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 3 to 4, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 4 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:40.394451 30673 ts_tablet_manager.cc:933] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:47:40.396492 30673 tablet_copy_client.cc:323] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.1:35499
I20260807 01:47:40.398824 27126 tablet_copy_service.cc:140] P 3262a9d183914b699227fdba0ca5bed3: Received BeginTabletCopySession request for tablet 1c4e145f8f52431fa41b1f596e2f1133 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:52556)
I20260807 01:47:40.399468 27126 tablet_copy_service.cc:161] P 3262a9d183914b699227fdba0ca5bed3: Beginning new tablet copy session on tablet 1c4e145f8f52431fa41b1f596e2f1133 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:52556: session id = 57bb4985157447f0a3ab680fa0026f12-1c4e145f8f52431fa41b1f596e2f1133
I20260807 01:47:40.408915 27126 tablet_copy_source_session.cc:215] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:40.412947 30673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c4e145f8f52431fa41b1f596e2f1133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:40.432456 30673 tablet_copy_client.cc:806] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:40.433286 30673 tablet_copy_client.cc:670] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:40.437022 30673 tablet_copy_client.cc:538] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:40.448426 30673 tablet_bootstrap.cc:492] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:40.497810 30673 tablet_bootstrap.cc:492] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:40.499024 30673 tablet_bootstrap.cc:492] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:40.499938 30673 ts_tablet_manager.cc:1403] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.052s user 0.034s sys 0.015s
I20260807 01:47:40.504082 30673 raft_consensus.cc:359] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:40.505051 30673 raft_consensus.cc:740] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:40.505867 30673 consensus_queue.cc:260] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:40.509224 30673 ts_tablet_manager.cc:1434] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260807 01:47:40.513566 27126 tablet_copy_service.cc:342] P 3262a9d183914b699227fdba0ca5bed3: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-1c4e145f8f52431fa41b1f596e2f1133 received from {username='slave'} at 127.0.0.1:52556
I20260807 01:47:40.514112 27126 tablet_copy_service.cc:434] P 3262a9d183914b699227fdba0ca5bed3: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-1c4e145f8f52431fa41b1f596e2f1133 on tablet 1c4e145f8f52431fa41b1f596e2f1133 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:40.581568 30673 ts_tablet_manager.cc:933] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:40.788684 30673 tablet_copy_client.cc:323] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.2:34855
I20260807 01:47:40.916442 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:41.628451 30688 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:41.629307 30688 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.636785 30688 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:41.639369 30688 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:41.640614 30688 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.638340 30693 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:41.638737 30699 raft_consensus.cc:493] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:41.644582 30693 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:41.645181 30699 raft_consensus.cc:515] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.660383 30683 raft_consensus.cc:493] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:41.661175 30683 raft_consensus.cc:515] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.662451 30696 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:41.663137 30696 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:41.668051 30699 leader_election.cc:290] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:41.668680 30688 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:41.669087 30704 raft_consensus.cc:493] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:41.669705 30693 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:41.670205 30704 raft_consensus.cc:515] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:41.671730 30690 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:41.672541 30690 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.673158 30704 leader_election.cc:290] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:41.668051 30696 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:41.675673 30690 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:41.676573 30693 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:41.669390 30683 leader_election.cc:290] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:41.702811 30712 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:41.703641 30712 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.703991 30713 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:41.705832 30713 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:41.706568 30712 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:41.709475 30713 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:41.682865 30704 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:41.698760 30683 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:41.693550 30693 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:41.717636 30683 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.719473 30693 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:41.720060 30683 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:41.720857 30704 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:41.723089 30704 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:41.723615 30704 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:41.724161 30704 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.708992 30712 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:41.723129 26952 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60866 (request call id 206) took 1210 ms. Trace:
I20260807 01:47:41.756424 30681 raft_consensus.cc:493] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:41.757236 30681 raft_consensus.cc:515] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:41.767469 30694 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:41.767244 30696 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:41.768080 30694 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:41.768520 30696 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.753929 26952 rpcz_store.cc:276] 0807 01:47:40.513153 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:40.513348 (+ 195us) service_pool.cc:224] Handling call
0807 01:47:41.723027 (+1209679us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":175275,"reported_tablets":1}
I20260807 01:47:41.770552 30694 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:41.770689 30681 leader_election.cc:290] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:41.766840 30690 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:41.755239 30712 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:41.759783 30704 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:41.771375 30696 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:41.772926 30715 raft_consensus.cc:493] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:41.777074 30715 raft_consensus.cc:515] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.775419 30716 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:41.780853 30716 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.778443 30712 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:41.783509 30716 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:41.778620 30686 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:41.786176 30686 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:41.789083 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:41.778393 30698 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:41.791131 30698 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.794039 30698 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:41.775414 30690 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:41.792049 30686 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:41.785414 30715 leader_election.cc:290] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:41.800863 30698 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:41.801607 30698 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.794734 30716 raft_consensus.cc:493] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:41.795140 30681 raft_consensus.cc:493] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:41.810561 30716 raft_consensus.cc:515] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.811378 30681 raft_consensus.cc:515] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.798813 30690 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:41.816560 30700 raft_consensus.cc:493] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:41.817212 30700 raft_consensus.cc:515] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.819825 30700 leader_election.cc:290] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:41.822826 30704 raft_consensus.cc:493] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:41.823565 30704 raft_consensus.cc:515] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.815354 30696 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:41.832871 30696 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.834534 30690 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:41.835317 30690 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.841890 30681 leader_election.cc:290] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:41.848623 30681 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:41.849360 30681 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:41.853699 30681 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:41.837916 30690 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:41.860656 30690 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:41.861886 30681 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:41.862771 30681 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.865087 30681 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:41.867790 30704 leader_election.cc:290] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:41.861373 30690 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:41.876385 30690 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:41.877660 30690 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:41.878372 30690 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.880654 30690 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:41.917282 30681 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:41.918169 30686 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:41.918799 30686 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:41.921180 30686 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:41.921772 30686 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:41.917896 30681 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.924309 30681 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:41.924991 30681 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:41.925673 30681 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:41.926579 30707 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:41.927134 30707 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:41.928085 30681 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:41.929320 30707 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:41.929896 30686 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:41.932356 30686 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:41.934404 30707 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:41.935423 30707 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:41.937809 30707 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:41.942448 30704 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:41.943260 30704 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.945781 30724 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:41.946524 30724 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:41.951682 30724 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:41.947644 30723 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:41.963112 30700 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:41.963264 30723 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:41.966444 30696 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:41.966964 30723 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:41.962447 30721 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:41.963797 30700 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:41.967250 30696 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:41.968386 30696 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:41.969478 30700 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:41.967890 30721 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.971012 30696 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:41.988821 30715 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:41.989291 30705 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:41.989898 30705 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.990527 30690 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:41.972281 30721 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:41.990799 30717 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:41.992903 30705 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:41.992776 30717 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.969595 30708 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:41.995276 30717 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:41.953828 30704 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:41.977747 30702 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:41.994239 30708 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:41.998509 30702 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.000590 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.001343 30702 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:47:42.002846 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.003458 27109 raft_consensus.cc:2468] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:47:42.003953 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.004456 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.004632 27230 raft_consensus.cc:2468] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:47:42.001803 30708 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.005785 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.008332 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:41.988701 30719 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:41.982653 30688 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:42.010015 30688 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.011036 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.011543 30717 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:42.011763 27105 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:41.978211 27099 raft_consensus.cc:1275] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260807 01:47:42.011972 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:41.984768 30699 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:42.012102 30717 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:42.012923 30699 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:41.991127 30690 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:42.013247 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.015147 30699 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.015475 30717 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:42.015702 30699 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:41.989467 30715 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:42.012166 30688 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.017812 30690 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.016417 30716 leader_election.cc:290] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:41.986496 30698 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:42.024667 30688 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:42.029055 30716 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:42.029501 30688 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.029662 30716 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:42.026322 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.027019 30705 raft_consensus.cc:493] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:42.027153 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.032982 30705 raft_consensus.cc:515] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.027545 30721 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:42.035213 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.035702 30721 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.038813 30690 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:42.039376 30690 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:42.039628 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.041062 30723 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:42.027933 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.037936 30721 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:42.041621 30723 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:42.039047 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.042371 30690 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.035184 30705 leader_election.cc:290] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:42.028083 30718 raft_consensus.cc:493] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:42.028393 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.043723 30723 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.025095 30715 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:42.046340 30700 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:42.043659 30718 raft_consensus.cc:515] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.046898 30700 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.047356 30723 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:42.047843 30723 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:42.048961 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.049603 27230 raft_consensus.cc:2468] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:47:42.049721 30723 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.051059 30718 leader_election.cc:290] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:41.990831 30711 raft_consensus.cc:493] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:42.032287 30688 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.055804 30688 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:42.045230 30720 raft_consensus.cc:493] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:42.049006 30700 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:42.056365 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.001883 30702 raft_consensus.cc:493] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:42.057056 30720 raft_consensus.cc:515] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.055619 30711 raft_consensus.cc:515] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.058056 30702 raft_consensus.cc:515] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.059798 30720 leader_election.cc:290] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:42.025583 30699 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.060483 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.061177 30711 leader_election.cc:290] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:42.060947 30688 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.069175 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.069798 30711 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:42.070317 30705 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:42.071470 30688 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:42.071889 30715 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:42.072629 30688 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:42.071555 30705 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:42.073206 30688 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.073844 30728 raft_consensus.cc:493] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:42.074429 30728 raft_consensus.cc:515] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.076187 30705 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:42.062954 30704 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:42.076813 30728 leader_election.cc:290] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:42.076993 30704 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:42.070554 30711 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:42.064109 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.065274 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.079263 30704 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.080885 30711 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:42.080102 27327 raft_consensus.cc:2468] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 2.
I20260807 01:47:42.080370 27337 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:47:42.082831 30726 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:42.080773 30690 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:42.072722 30715 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:42.083124 30734 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:42.086045 30734 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.083423 30726 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.087491 30715 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:42.066202 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.089110 27222 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:47:42.089195 30707 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:42.067194 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.089843 28715 raft_consensus.cc:1217] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEARNER]: Deduplicated request from leader. Original: 5.4->[5.5-5.5] Dedup: 5.5->[]
I20260807 01:47:42.089780 30707 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.067565 30708 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:42.092154 30708 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.094821 30735 raft_consensus.cc:493] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:42.018589 30719 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:42.105991 30719 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:42.060500 30702 leader_election.cc:290] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:42.106806 30702 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:42.107373 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.107338 30702 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.106837 30735 raft_consensus.cc:515] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.109711 30702 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:42.076395 30688 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.062191 30699 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.063388 30724 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:42.111543 30724 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.111828 30715 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:42.065497 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.114125 30715 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:42.114645 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.115636 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.115751 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.116235 30715 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:42.088544 30734 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.092593 30707 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:42.062971 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.117105 30696 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
W20260807 01:47:42.093502 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 46800.7us
I20260807 01:47:42.118585 27172 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:42.117233 27337 raft_consensus.cc:1275] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:47:42.067693 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.088305 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.120853 30724 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:42.121426 30724 raft_consensus.cc:2804] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:47:42.121778 30724 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:42.122133 30724 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:42.122341 30700 consensus_queue.cc:1048] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 01:47:42.123364 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.095952 30708 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:42.124248 27172 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:42.117662 30696 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:42.124996 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.097397 30728 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:42.125921 30728 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.126955 30696 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.127960 30700 raft_consensus.cc:2804] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:47:42.128103 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.128391 30700 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:42.128750 28709 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:47:42.128774 30700 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:42.099114 30726 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.134712 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.136544 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.137291 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.137970 27338 raft_consensus.cc:2468] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:47:42.138392 30728 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.139310 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.139925 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.140633 30700 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:42.096107 30736 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:42.046115 30716 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:42.110131 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.141536 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.111750 30727 raft_consensus.cc:493] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:42.088120 30690 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:42.143389 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.143935 30690 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.101246 30737 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:42.145212 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.146344 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.146718 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.145030 30727 raft_consensus.cc:515] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:42.109959 30735 leader_election.cc:290] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.147054 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.142668 30736 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.148221 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.148682 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.151845 27102 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:47:42.152324 28711 raft_consensus.cc:2468] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:47:42.152660 27336 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:47:42.146387 27338 raft_consensus.cc:2468] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:47:42.151772 30727 leader_election.cc:290] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:42.155126 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.155267 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.157632 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.160179 28710 raft_consensus.cc:2468] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:47:42.154771 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.158849 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.161360 27338 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:42.164232 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.164657 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.166236 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.166468 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.167970 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.168641 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.168849 27338 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
I20260807 01:47:42.170302 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.184266 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.188870 30733 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:42.183403 30690 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:42.194646 30690 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.195653 30700 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.197835 27229 raft_consensus.cc:1275] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 01:47:42.198043 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.199326 30690 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:42.201057 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.203235 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.204939 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.205124 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.205811 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.206490 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.203135 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.205014 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.207144 27222 raft_consensus.cc:2468] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 2.
I20260807 01:47:42.207682 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.206904 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.208957 27328 raft_consensus.cc:2468] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:47:42.209249 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.207780 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.192022 30696 raft_consensus.cc:493] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:42.206375 27246 tablet_copy_service.cc:140] P 3478729505b046c2a0051c2d0130873f: Received BeginTabletCopySession request for tablet 00395257beb442e892a4ec18c4c30d95 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:41230)
I20260807 01:47:42.219301 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.219256 30739 ts_tablet_manager.cc:933] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:47:42.220732 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.221453 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.222790 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.223598 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.227700 27175 leader_election.cc:304] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:42.217341 27172 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:42.228994 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.231222 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.229285 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.235495 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.238116 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.240180 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.240175 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.242743 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.242971 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.242839 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.258946 27336 raft_consensus.cc:2468] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
I20260807 01:47:42.250028 28706 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:42.257073 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.256868 27175 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:42.268374 28710 raft_consensus.cc:2468] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:47:42.260207 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.269037 27175 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:42.264992 27067 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:42.261910 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.269739 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.270046 27175 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:42.262082 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.264564 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.260121 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.271514 27175 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:42.267534 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.272684 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.273167 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.271267 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.274353 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.271924 27328 raft_consensus.cc:2468] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 1.
I20260807 01:47:42.145941 30737 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.278568 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.278723 30731 raft_consensus.cc:2804] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:47:42.279196 30723 raft_consensus.cc:2804] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:47:42.281610 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.279876 28709 raft_consensus.cc:1275] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260807 01:47:42.278198 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:42.279996 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.279956 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.283424 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.266039 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.276770 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.287511 30725 raft_consensus.cc:2749] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:42.289079 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.290364 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.291796 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.291693 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.294888 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.297766 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.298027 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.298287 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.297384 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.300527 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.260010 30708 raft_consensus.cc:2749] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:42.272197 30693 raft_consensus.cc:2804] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:47:42.304339 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.256500 27246 tablet_copy_service.cc:161] P 3478729505b046c2a0051c2d0130873f: Beginning new tablet copy session on tablet 00395257beb442e892a4ec18c4c30d95 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:41230: session id = 57bb4985157447f0a3ab680fa0026f12-00395257beb442e892a4ec18c4c30d95
I20260807 01:47:42.304930 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.308626 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.311228 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.312582 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.304733 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.235376 30733 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.288661 30720 raft_consensus.cc:2749] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:42.318439 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.291720 30723 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:42.294705 30714 raft_consensus.cc:2804] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:47:42.319501 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.319605 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.315851 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.317970 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.322942 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.328821 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.323318 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.246402 30696 raft_consensus.cc:515] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.294587 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.332079 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.332595 27063 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:42.279202 30731 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:42.325196 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.330905 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.333302 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.324842 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.342684 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.339196 30714 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:42.308086 30693 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
W20260807 01:47:42.351481 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 68544.3us
I20260807 01:47:42.351554 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.347100 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.355705 27063 leader_election.cc:304] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:42.356663 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.357335 27230 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:47:42.342072 27229 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:47:42.359841 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:47:42.340047 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 43481.1us
I20260807 01:47:42.300880 30704 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:42.297394 30736 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.332923 30696 leader_election.cc:290] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:42.330753 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.361302 30681 consensus_queue.cc:1048] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260807 01:47:42.330761 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.343881 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.371683 27214 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:47:42.372583 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.373777 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.385897 30724 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:42.387285 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:47:42.344455 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 48643.2us
I20260807 01:47:42.340437 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.342577 27226 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:42.346408 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.362581 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.389846 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.391304 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.392046 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.392431 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.393144 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.343408 30731 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:42.366225 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.366645 30733 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:42.367785 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.361472 30683 raft_consensus.cc:2749] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:47:42.376778 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.326520 30723 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:42.358345 30714 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:42.377436 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.378993 30704 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:42.366284 30738 ts_tablet_manager.cc:933] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:42.404280 27216 raft_consensus.cc:2468] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:47:42.380738 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.404822 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.381770 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.382270 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:42.382514 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.383302 30737 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:42.383435 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.362998 30693 raft_consensus.cc:3060] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:42.333796 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.385340 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.386042 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.409910 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.390127 27287 leader_election.cc:304] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:42.398447 27214 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:42.417102 27214 raft_consensus.cc:606] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:42.363986 28655 leader_election.cc:304] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:42.420553 30716 raft_consensus.cc:2804] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:42.425251 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:47:42.427173 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 71555us
W20260807 01:47:42.427913 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 63965.8us
I20260807 01:47:42.419688 27287 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:42.428550 27063 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:42.428658 28655 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:42.430769 28655 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:42.424351 28660 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:47:42.432999 28660 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:42.433589 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:42.434329 28660 leader_election.cc:304] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:42.436211 28660 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:42.437829 30726 raft_consensus.cc:2749] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:42.438647 30735 raft_consensus.cc:2804] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:42.438957 30694 raft_consensus.cc:2749] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:42.440405 30704 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.425145 30716 raft_consensus.cc:493] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:42.441273 30716 raft_consensus.cc:3060] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:42.446476 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.447387 27246 tablet_copy_source_session.cc:215] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:42.450344 30687 raft_consensus.cc:2804] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:47:42.450855 30687 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:42.451241 30687 raft_consensus.cc:3060] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:42.455716 28660 leader_election.cc:304] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:42.456938 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.459520 28660 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:42.461072 27289 leader_election.cc:304] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:42.461858 30702 raft_consensus.cc:2749] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:42.464210 30738 tablet_copy_client.cc:323] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.2:34855
I20260807 01:47:42.459857 28655 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:42.465694 28655 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:42.458153 27287 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:42.467378 28655 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:47:42.465579 30702 raft_consensus.cc:2804] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:47:42.470045 27063 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:42.470412 27175 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:42.472324 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.472781 27289 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:42.473649 27175 leader_election.cc:304] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:42.452033 30715 raft_consensus.cc:2804] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:42.475557 27063 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:42.454838 30688 raft_consensus.cc:2804] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:47:42.454523 30728 raft_consensus.cc:2749] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:42.466969 30726 raft_consensus.cc:2749] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:42.492812 30732 raft_consensus.cc:2749] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:42.454089 30735 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:42.468647 30734 raft_consensus.cc:2749] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:47:42.467258 30737 raft_consensus.cc:2749] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:47:42.478570 30717 raft_consensus.cc:2749] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:42.509316 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.468490 30702 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:42.519886 30702 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Advancing to term 8
W20260807 01:47:42.520862 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 57531.2us
I20260807 01:47:42.522948 30698 raft_consensus.cc:2804] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:47:42.523106 30739 tablet_copy_client.cc:323] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Beginning tablet copy session from remote peer at address 127.24.4.1:35499
I20260807 01:47:42.523507 30698 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:42.488411 30686 raft_consensus.cc:2749] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:47:42.471793 30699 raft_consensus.cc:2749] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:42.474933 30704 raft_consensus.cc:2804] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:42.491415 27063 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:42.487241 30707 raft_consensus.cc:2804] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260807 01:47:42.510953 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 43658.2us
I20260807 01:47:42.498625 30712 raft_consensus.cc:2804] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:47:42.534003 30712 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:42.534489 30712 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:42.495983 30715 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:42.528759 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.497202 30688 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:42.507067 30736 raft_consensus.cc:2804] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:47:42.506098 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.548429 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.532758 27063 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:42.532864 28655 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:42.535411 28660 leader_election.cc:304] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:42.514632 30735 raft_consensus.cc:3060] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:42.518635 30724 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.488560 30733 consensus_queue.cc:1048] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260807 01:47:42.526031 30698 raft_consensus.cc:3060] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:42.543015 27105 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0e280c976844349b4874a479aa08de2"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:47:42.543506 30688 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:42.538767 30715 raft_consensus.cc:3060] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:42.532290 30707 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:42.534231 30690 raft_consensus.cc:2804] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:42.547783 30716 raft_consensus.cc:515] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.555982 27105 raft_consensus.cc:606] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:42.555995 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:42.554507 30749 raft_consensus.cc:2749] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:42.530874 30704 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:42.560038 30742 raft_consensus.cc:2804] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:42.547488 30714 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.556778 27067 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
W20260807 01:47:42.565359 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 88809us
I20260807 01:47:42.552994 28660 leader_election.cc:304] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:42.590296 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.568133 30690 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:42.578761 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.567150 30707 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:42.567685 30694 raft_consensus.cc:2804] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:47:42.566064 30726 raft_consensus.cc:2749] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260807 01:47:42.583392 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 125290us
I20260807 01:47:42.586786 27289 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:42.594481 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260807 01:47:42.609426 27289 leader_election.cc:304] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:42.610299 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:47:42.610869 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:42.598459 27126 tablet_copy_service.cc:140] P 3262a9d183914b699227fdba0ca5bed3: Received BeginTabletCopySession request for tablet 67fed460b08e4124a6d4484db52478b1 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:52556)
I20260807 01:47:42.578312 30704 raft_consensus.cc:3060] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:42.602653 28660 leader_election.cc:304] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:42.613759 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:42.627508 27289 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:42.610908 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.632714 28660 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:42.642916 27287 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:42.644001 27287 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:42.646042 30721 raft_consensus.cc:2804] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:47:42.646852 30721 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:42.647307 30721 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:42.604652 30690 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:42.653592 27287 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:42.607612 30681 raft_consensus.cc:2804] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:42.651418 27126 tablet_copy_service.cc:161] P 3262a9d183914b699227fdba0ca5bed3: Beginning new tablet copy session on tablet 67fed460b08e4124a6d4484db52478b1 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:52556: session id = 57bb4985157447f0a3ab680fa0026f12-67fed460b08e4124a6d4484db52478b1
I20260807 01:47:42.612468 30686 raft_consensus.cc:2749] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:42.660152 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:42.608434 30694 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:42.610829 30705 raft_consensus.cc:2749] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:47:42.657240 30730 raft_consensus.cc:2749] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:42.659194 28660 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:42.609941 30742 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:42.651342 30711 raft_consensus.cc:2749] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:47:42.653076 30728 raft_consensus.cc:2749] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:42.657416 30750 raft_consensus.cc:2749] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:42.652259 27246 tablet_copy_service.cc:140] P 3478729505b046c2a0051c2d0130873f: Received BeginTabletCopySession request for tablet de3b20c9b4a7467ea9e6c098167df7e3 from peer 57bb4985157447f0a3ab680fa0026f12 ({username='slave'} at 127.0.0.1:41230)
I20260807 01:47:42.658784 30681 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:42.670830 30694 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:47:42.644608 30687 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:42.656126 30731 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:42.673694 30693 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.671551 30726 raft_consensus.cc:2749] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:47:42.704802 27175 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:42.708045 27126 tablet_copy_source_session.cc:215] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:42.713647 30687 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.714493 30733 raft_consensus.cc:2749] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:42.680815 30705 raft_consensus.cc:2804] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:47:42.657284 30757 raft_consensus.cc:2804] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:47:42.707185 30702 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.674075 30742 raft_consensus.cc:3060] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:42.689445 30681 raft_consensus.cc:3060] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
W20260807 01:47:42.714005 27373 debug-util.cc:398] Leaking SignalData structure 0x7b080016a380 after lost signal to thread 24600
W20260807 01:47:42.725070 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 71557.2us
I20260807 01:47:42.730189 27287 leader_election.cc:304] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:42.732088 27064 leader_election.cc:304] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:42.736744 27064 leader_election.cc:304] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:42.737496 27287 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:42.698612 27246 tablet_copy_service.cc:161] P 3478729505b046c2a0051c2d0130873f: Beginning new tablet copy session on tablet de3b20c9b4a7467ea9e6c098167df7e3 from peer 57bb4985157447f0a3ab680fa0026f12 at {username='slave'} at 127.0.0.1:41230: session id = 57bb4985157447f0a3ab680fa0026f12-de3b20c9b4a7467ea9e6c098167df7e3
I20260807 01:47:42.742520 27287 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:42.725445 30687 consensus_queue.cc:1048] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260807 01:47:42.745321 27373 debug-util.cc:398] Leaking SignalData structure 0x7b0800190d40 after lost signal to thread 27011
I20260807 01:47:42.746809 27287 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
W20260807 01:47:42.747851 27373 debug-util.cc:398] Leaking SignalData structure 0x7b08001e63c0 after lost signal to thread 27146
W20260807 01:47:42.749598 27373 debug-util.cc:398] Leaking SignalData structure 0x7b08001c77e0 after lost signal to thread 27269
W20260807 01:47:42.751111 27373 debug-util.cc:398] Leaking SignalData structure 0x7b0800323180 after lost signal to thread 27378
W20260807 01:47:42.752521 27373 debug-util.cc:398] Leaking SignalData structure 0x7b08000640c0 after lost signal to thread 28748
I20260807 01:47:42.758373 30690 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:42.760881 30690 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.761514 30690 raft_consensus.cc:2749] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:42.721659 30705 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 01:47:42.763376 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 118080us
I20260807 01:47:42.764655 27289 leader_election.cc:304] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:42.764544 27287 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:42.765923 30753 raft_consensus.cc:2749] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:47:42.767089 30769 raft_consensus.cc:2804] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:47:42.767516 27287 leader_election.cc:304] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:42.770455 27287 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:42.770537 27289 leader_election.cc:304] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:42.767700 30769 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:42.735747 30757 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:42.762578 30711 raft_consensus.cc:2749] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:47:42.770304 30686 raft_consensus.cc:2749] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:42.765029 30705 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:42.769973 30719 raft_consensus.cc:2749] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:42.775854 30753 raft_consensus.cc:2749] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:42.782538 30716 leader_election.cc:290] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:42.796263 30753 raft_consensus.cc:2749] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:42.797734 30736 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:42.799641 30688 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.801903 30723 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.794852 30718 raft_consensus.cc:2749] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:47:42.796269 28707 raft_consensus.cc:1240] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 7. Current term is 8. Ops: []
I20260807 01:47:42.782601 30769 raft_consensus.cc:3060] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:42.789075 30764 raft_consensus.cc:2749] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:42.812263 30702 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.786348 30757 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:42.790220 30711 raft_consensus.cc:2749] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:47:42.822288 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:42.827008 30735 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.828320 27246 tablet_copy_source_session.cc:215] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 01:47:42.812920 30714 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:42.852628 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:42.820863 30736 raft_consensus.cc:3060] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:42.856956 27333 raft_consensus.cc:1240] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:42.859755 30723 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:42.809768 30698 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:42.843528 30708 consensus_queue.cc:1059] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260807 01:47:42.882606 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:42.867882 30712 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:42.892004 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:42.894304 27175 leader_election.cc:304] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:42.895655 30708 raft_consensus.cc:2749] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:42.900256 30712 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.883596 30735 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.886612 30715 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:42.903563 30715 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:42.904101 28655 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:42.904348 30688 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.904999 30688 raft_consensus.cc:2749] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:47:42.910351 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
W20260807 01:47:42.920974 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 184559us
I20260807 01:47:42.922372 27064 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12
W20260807 01:47:42.922858 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 152309us
I20260807 01:47:42.925020 27289 leader_election.cc:304] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:42.917989 30702 consensus_queue.cc:1059] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260807 01:47:42.931565 28660 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:42.932962 27289 leader_election.cc:304] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:42.930384 30702 raft_consensus.cc:3055] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:42.925314 30698 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:42.928792 30764 raft_consensus.cc:2804] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:42.923949 27064 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:42.928032 30707 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:42.954411 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:42.938087 30702 raft_consensus.cc:740] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:42.957171 30714 raft_consensus.cc:2749] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:47:42.942191 30699 raft_consensus.cc:2749] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260807 01:47:42.963275 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:42.964954 30707 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:42.954005 27064 leader_election.cc:304] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:42.967242 30673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00395257beb442e892a4ec18c4c30d95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:42.972376 30694 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:42.941247 30719 raft_consensus.cc:2749] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:42.978647 30723 raft_consensus.cc:3055] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 7 LEADER]: Stepping down as leader of term 7
I20260807 01:47:42.979146 30723 raft_consensus.cc:740] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:42.979365 30736 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.984321 30702 consensus_queue.cc:260] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.985540 30704 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:42.942266 30686 raft_consensus.cc:2749] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:42.952076 30764 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:42.989348 30764 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:42.991927 27064 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:42.994581 28714 raft_consensus.cc:1240] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 4. Current term is 5. Ops: []
I20260807 01:47:42.987457 30702 raft_consensus.cc:3060] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:43.002815 28707 raft_consensus.cc:1240] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 1. Current term is 2. Ops: []
I20260807 01:47:43.005234 30704 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:43.007850 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
W20260807 01:47:43.009507 28655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:42.951251 30681 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:43.024609 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:43.001271 30690 raft_consensus.cc:2749] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:47:42.990645 30705 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:43.028025 30763 raft_consensus.cc:2749] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:43.028195 28707 raft_consensus.cc:1240] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:42.976351 30757 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:43.021888 30694 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:43.035344 30681 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:43.006793 30704 consensus_queue.cc:1059] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260807 01:47:43.040946 30757 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:43.042402 30693 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:47:43.045293 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 47356.4us
I20260807 01:47:43.046466 27064 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:43.042146 30743 consensus_queue.cc:1059] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
W20260807 01:47:43.047879 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 277483us
I20260807 01:47:43.048671 30731 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:43.048744 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:43.051345 27064 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.053556 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:43.007946 30742 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:43.055480 27287 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:43.056478 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:43.061955 27287 leader_election.cc:304] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:43.062597 30738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de3b20c9b4a7467ea9e6c098167df7e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:43.068832 30727 raft_consensus.cc:2749] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:47:43.069473 27170 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.070473 30694 raft_consensus.cc:2749] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:43.073758 30763 raft_consensus.cc:3055] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:43.077962 30763 raft_consensus.cc:740] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:43.078197 30742 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:43.078614 30704 raft_consensus.cc:2749] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:43.075441 30705 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:43.075740 27063 leader_election.cc:304] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:43.089543 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:43.090265 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:43.090487 27064 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.092415 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:43.091837 27064 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.094992 27226 raft_consensus.cc:1240] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 5. Current term is 6. Ops: []
I20260807 01:47:43.046567 30736 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:43.096206 27103 raft_consensus.cc:1240] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:43.101861 27212 raft_consensus.cc:1240] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 6. Current term is 7. Ops: []
I20260807 01:47:43.097683 27226 raft_consensus.cc:1240] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 5. Current term is 6. Ops: []
I20260807 01:47:43.103140 27170 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.108898 27170 leader_election.cc:304] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.109285 30690 raft_consensus.cc:2749] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:43.110193 27170 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.110966 30687 raft_consensus.cc:2804] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:47:43.111406 30687 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:43.111770 30687 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:43.112082 30696 raft_consensus.cc:2749] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:43.112841 30696 raft_consensus.cc:2749] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:43.113459 30696 raft_consensus.cc:2749] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:47:43.082818 30763 consensus_queue.cc:260] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:43.114866 27101 raft_consensus.cc:1240] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 6. Current term is 7. Ops: []
I20260807 01:47:43.104009 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:43.104220 27229 raft_consensus.cc:1240] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 1. Current term is 2. Ops: []
I20260807 01:47:43.115926 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:43.117100 27170 leader_election.cc:304] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.069895 30767 raft_consensus.cc:2749] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:43.076102 28706 raft_consensus.cc:1240] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:43.055900 30681 raft_consensus.cc:2749] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:43.101811 30694 raft_consensus.cc:2749] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:47:43.118067 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:43.029882 30723 consensus_queue.cc:260] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:43.126102 30725 raft_consensus.cc:3055] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:43.126600 30725 raft_consensus.cc:740] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:43.130399 27064 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.131772 27064 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.133581 27170 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.136264 27064 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:43.136719 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:43.127496 30725 consensus_queue.cc:260] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:43.137270 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:43.138208 30725 raft_consensus.cc:3060] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:43.138950 28655 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.142983 27214 raft_consensus.cc:1240] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:43.145339 27337 raft_consensus.cc:1240] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260807 01:47:43.146355 27064 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:43.147439 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:43.148056 30760 raft_consensus.cc:2749] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:47:43.148870 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:43.150241 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:43.060993 30721 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:43.152622 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:43.154191 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:43.155119 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:43.159610 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:43.161043 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:43.156572 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:43.156857 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:43.121513 30769 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:43.159562 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:43.165186 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:43.165603 27170 leader_election.cc:304] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.139197 30732 raft_consensus.cc:2749] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:43.166307 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:43.167002 27326 raft_consensus.cc:1240] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 1. Current term is 2. Ops: []
I20260807 01:47:43.170527 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:43.170519 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:43.174046 27170 leader_election.cc:304] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.175256 27170 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.140610 30707 raft_consensus.cc:2749] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:43.141166 30681 raft_consensus.cc:2749] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:47:43.187549 27330 raft_consensus.cc:1240] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 3. Current term is 4. Ops: []
I20260807 01:47:43.177233 30693 raft_consensus.cc:2749] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:43.184195 30696 raft_consensus.cc:2749] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:43.143169 30708 raft_consensus.cc:2749] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:43.156567 30757 consensus_queue.cc:1059] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260807 01:47:43.155121 30778 raft_consensus.cc:2749] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:43.190533 30769 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:43.208503 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:43.210094 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:43.210161 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
W20260807 01:47:43.207613 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 207674us
I20260807 01:47:43.227557 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:43.229671 27170 leader_election.cc:304] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.212828 30714 raft_consensus.cc:2749] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:43.234515 27170 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.236219 27170 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.217449 30704 raft_consensus.cc:2749] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:47:43.245658 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:43.242594 27287 leader_election.cc:304] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:43.254307 30694 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:43.255089 30694 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:43.263361 30763 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:43.267402 30694 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:43.240881 30720 raft_consensus.cc:2749] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 01:47:43.247308 30757 consensus_queue.cc:1059] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260807 01:47:43.238415 30693 raft_consensus.cc:2749] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:43.268405 30708 raft_consensus.cc:2749] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:43.288499 30778 raft_consensus.cc:493] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:43.238032 30717 consensus_queue.cc:1059] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260807 01:47:43.289835 30778 raft_consensus.cc:515] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:43.278515 30694 consensus_queue.cc:1059] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:47:43.279693 30727 raft_consensus.cc:2749] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:47:43.293066 30723 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:47:43.280761 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:43.213348 30711 consensus_queue.cc:1059] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260807 01:47:43.295930 30711 raft_consensus.cc:3055] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:47:43.296470 30711 raft_consensus.cc:740] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:43.283774 30739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67fed460b08e4124a6d4484db52478b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 01:47:43.240280 30764 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:43.245571 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:43.284821 30757 raft_consensus.cc:3055] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:43.287611 30687 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:43.302050 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 59681.5us
I20260807 01:47:43.316748 30727 consensus_queue.cc:1059] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:43.318406 30727 raft_consensus.cc:3055] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:47:43.318825 30727 raft_consensus.cc:740] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:43.297607 30711 consensus_queue.cc:260] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:43.321053 30711 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:43.311151 30757 raft_consensus.cc:740] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
W20260807 01:47:43.331770 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 157118us
I20260807 01:47:43.330704 30769 consensus_queue.cc:1059] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260807 01:47:43.334257 30778 leader_election.cc:290] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:43.335314 30721 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:43.340350 27287 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:43.349002 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:43.349905 27063 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.319950 30727 consensus_queue.cc:260] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:43.391247 30757 consensus_queue.cc:260] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:43.392927 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:43.393823 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:47:43.427593 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 87422.1us
I20260807 01:47:43.544063 27287 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
W20260807 01:47:43.548265 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 193542us
I20260807 01:47:43.551440 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:43.551528 27103 raft_consensus.cc:1240] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:43.548666 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:43.413995 30686 raft_consensus.cc:3055] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:43.503556 30683 raft_consensus.cc:3055] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:43.504292 30753 raft_consensus.cc:2749] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:47:43.507299 28713 raft_consensus.cc:1240] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:43.510245 30778 raft_consensus.cc:2749] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260807 01:47:43.512075 30687 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:43.513657 30727 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:43.514462 30764 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:43.521205 30681 raft_consensus.cc:493] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:47:43.523947 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 287800us
I20260807 01:47:43.525491 30757 raft_consensus.cc:3060] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:43.528609 27214 raft_consensus.cc:1240] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5. Ops: []
I20260807 01:47:43.406934 30717 raft_consensus.cc:3055] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Stepping down as leader of term 6
I20260807 01:47:43.499922 30743 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:43.551703 30776 raft_consensus.cc:2749] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:47:43.580395 28711 raft_consensus.cc:1240] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:43.581054 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:43.553212 30767 consensus_queue.cc:1059] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260807 01:47:43.589267 27170 leader_election.cc:304] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.560714 30686 raft_consensus.cc:740] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:43.564154 30683 raft_consensus.cc:740] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:43.553463 30769 consensus_queue.cc:1059] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260807 01:47:43.606134 27170 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.606599 27105 raft_consensus.cc:1240] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:43.607439 27170 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.609757 30789 raft_consensus.cc:1064] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:43.611163 27103 raft_consensus.cc:1240] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 3. Current term is 4. Ops: []
I20260807 01:47:43.588551 30717 raft_consensus.cc:740] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:43.611097 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:43.612593 30725 raft_consensus.cc:2749] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:43.616484 28660 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.626927 27105 raft_consensus.cc:1240] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 1. Current term is 2. Ops: []
I20260807 01:47:43.628619 27100 raft_consensus.cc:1240] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 4. Current term is 5. Ops: []
I20260807 01:47:43.587849 30681 raft_consensus.cc:515] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:43.634613 30779 raft_consensus.cc:3055] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:43.631819 30742 raft_consensus.cc:2749] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:47:43.638418 27101 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
I20260807 01:47:43.612757 30720 raft_consensus.cc:2749] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:43.640893 30683 consensus_queue.cc:260] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:43.614589 30723 raft_consensus.cc:2749] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 01:47:43.587906 30743 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:43.645829 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 84745.4us
I20260807 01:47:43.650844 30717 consensus_queue.cc:260] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:43.652145 30717 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:43.653854 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:43.654626 27063 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.656919 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:43.657435 30743 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:43.657375 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:43.656853 30738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:48350 (request call id 692) took 1513 ms. Trace:
I20260807 01:47:43.658615 30687 consensus_queue.cc:1059] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:43.659673 30687 raft_consensus.cc:3055] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:43.661387 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:43.662458 30686 consensus_queue.cc:260] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:43.663763 30686 raft_consensus.cc:3060] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:43.659240 30738 rpcz_store.cc:276] 0807 01:47:42.143018 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:42.143164 (+ 146us) service_pool.cc:224] Handling call
0807 01:47:42.426733 (+283569us) ts_tablet_manager.cc:934] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
0807 01:47:43.055233 (+628500us) tablet_copy_client.cc:368] Tablet copy session begun
0807 01:47:43.502996 (+447763us) tablet_metadata.cc:665] Metadata flushed
0807 01:47:43.510316 (+ 7320us) tablet_copy_client.cc:497] Wrote new tablet metadata
0807 01:47:43.637340 (+127024us) tablet_copy_client.cc:418] Wrote new consensus metadata
0807 01:47:43.637659 (+ 319us) tablet_replica.cc:206] Creating consensus instance
0807 01:47:43.645874 (+ 8215us) ts_tablet_manager.cc:989] Replica registered.
0807 01:47:43.656760 (+ 10886us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":316,"tablet-copy.queue_time_us":134711,"thread_start_us":82630,"threads_started":2}
I20260807 01:47:43.655627 30694 consensus_queue.cc:1059] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260807 01:47:43.649910 27170 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.664116 30789 consensus_queue.cc:237] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 3, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:43.668506 27212 raft_consensus.cc:1240] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:43.669035 27222 raft_consensus.cc:1240] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 6. Current term is 7. Ops: []
I20260807 01:47:43.669919 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:43.673089 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:43.673614 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:43.673848 30781 consensus_queue.cc:1059] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:43.679112 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:43.675453 30694 raft_consensus.cc:3055] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:43.682739 27109 raft_consensus.cc:1240] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 7. Current term is 8. Ops: []
I20260807 01:47:43.657981 30743 raft_consensus.cc:2749] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:47:43.683149 30681 leader_election.cc:290] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:43.683636 27105 raft_consensus.cc:1240] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 1. Current term is 2. Ops: []
I20260807 01:47:43.684939 30693 consensus_queue.cc:1059] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:43.692564 30711 raft_consensus.cc:3055] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Stepping down as leader of term 6
I20260807 01:47:43.687050 30731 raft_consensus.cc:993] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
I20260807 01:47:43.717725 30731 raft_consensus.cc:1081] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
W20260807 01:47:43.664949 30769 consensus_queue.cc:1175] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "57bb4985157447f0a3ab680fa0026f12" responder_term: 3 status { last_received { term: 2 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260807 01:47:43.647054 30779 raft_consensus.cc:740] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:43.700395 30783 raft_consensus.cc:2749] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:47:43.720399 30779 consensus_queue.cc:260] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:43.692469 30687 raft_consensus.cc:740] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:43.689103 30673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:48350 (request call id 679) took 3117 ms. Trace:
I20260807 01:47:43.649863 30778 consensus_queue.cc:1059] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260807 01:47:43.675392 30738 tablet_copy_client.cc:806] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:43.699652 27064 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.701606 28660 leader_election.cc:304] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.708103 27338 raft_consensus.cc:1240] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6. Ops: []
I20260807 01:47:43.708825 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:43.700343 30708 consensus_queue.cc:1059] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
I20260807 01:47:43.709110 27327 raft_consensus.cc:1240] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:43.712370 30781 raft_consensus.cc:3055] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:43.710942 27337 raft_consensus.cc:1240] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 4. Current term is 5. Ops: []
I20260807 01:47:43.712598 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:43.702893 30694 raft_consensus.cc:740] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:43.714419 30693 raft_consensus.cc:3055] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:43.715961 30711 raft_consensus.cc:740] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:43.702945 30725 consensus_queue.cc:1059] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260807 01:47:43.703009 30787 consensus_queue.cc:1059] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260807 01:47:43.710412 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:43.692773 30739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:52322 (request call id 514) took 1556 ms. Trace:
I20260807 01:47:43.713832 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:43.691500 27170 leader_election.cc:304] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.722873 30687 consensus_queue.cc:260] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:43.713464 30702 consensus_queue.cc:1059] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260807 01:47:43.722093 30779 raft_consensus.cc:3060] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:43.729305 30683 raft_consensus.cc:3060] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:43.729622 27064 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
W20260807 01:47:43.768335 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 76918.7us
I20260807 01:47:43.769122 27064 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.748087 28660 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.771612 27064 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.713565 27287 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:43.769122 27170 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.773226 27064 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.763804 30702 raft_consensus.cc:3055] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:43.732888 30708 raft_consensus.cc:3055] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:43.775758 30730 raft_consensus.cc:2804] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:47:43.776950 30708 raft_consensus.cc:740] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:43.777305 30730 raft_consensus.cc:697] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:43.777848 30708 consensus_queue.cc:260] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:43.778296 30730 consensus_queue.cc:237] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:43.779141 30708 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:43.758750 30687 raft_consensus.cc:3060] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:43.740146 30693 raft_consensus.cc:740] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
W20260807 01:47:43.776520 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 63109.4us
I20260807 01:47:43.787231 30693 consensus_queue.cc:260] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:43.788281 30693 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:43.788214 27287 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:43.750727 30711 consensus_queue.cc:260] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:43.791702 30711 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:43.793057 30790 raft_consensus.cc:2749] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 01:47:43.739459 30673 rpcz_store.cc:276] 0807 01:47:40.571773 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:40.572223 (+ 450us) service_pool.cc:224] Handling call
0807 01:47:40.581974 (+ 9751us) ts_tablet_manager.cc:934] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
0807 01:47:42.939545 (+2357571us) tablet_copy_client.cc:368] Tablet copy session begun
0807 01:47:43.181780 (+242235us) tablet_metadata.cc:665] Metadata flushed
0807 01:47:43.181910 (+ 130us) tablet_copy_client.cc:497] Wrote new tablet metadata
0807 01:47:43.688262 (+506352us) tablet_copy_client.cc:418] Wrote new consensus metadata
0807 01:47:43.688576 (+ 314us) tablet_replica.cc:206] Creating consensus instance
0807 01:47:43.688946 (+ 370us) ts_tablet_manager.cc:989] Replica registered.
0807 01:47:43.689018 (+ 72us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":238,"tablet-copy.queue_time_us":8229,"thread_start_us":204753,"threads_started":1}
I20260807 01:47:43.740131 30736 raft_consensus.cc:2749] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:47:43.750240 30725 raft_consensus.cc:3055] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:43.739969 30694 consensus_queue.cc:260] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:43.775820 30783 raft_consensus.cc:2749] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260807 01:47:43.729362 30712 consensus_queue.cc:1175] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "fcf4e45defb94276bcfa68392711dade" responder_term: 2 status { last_received { term: 2 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 2 index: 5 } }
I20260807 01:47:43.733103 30781 raft_consensus.cc:740] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:43.728681 30738 tablet_copy_client.cc:670] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:43.729792 30764 raft_consensus.cc:2749] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:47:43.770478 30731 raft_consensus.cc:2749] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:43.772230 30743 raft_consensus.cc:2749] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:47:43.795650 30782 raft_consensus.cc:2749] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260807 01:47:43.750511 30787 raft_consensus.cc:3055] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:43.802397 30787 raft_consensus.cc:740] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:43.772512 30778 raft_consensus.cc:2749] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:47:43.776190 30681 raft_consensus.cc:2749] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 01:47:43.730239 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:43.757141 30739 rpcz_store.cc:276] 0807 01:47:42.136065 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:42.136242 (+ 177us) service_pool.cc:224] Handling call
0807 01:47:42.246120 (+109878us) ts_tablet_manager.cc:934] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12: Initiating tablet copy from peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
0807 01:47:43.214826 (+968706us) tablet_copy_client.cc:368] Tablet copy session begun
0807 01:47:43.622533 (+407707us) tablet_metadata.cc:665] Metadata flushed
0807 01:47:43.632772 (+ 10239us) tablet_copy_client.cc:497] Wrote new tablet metadata
0807 01:47:43.692016 (+ 59244us) tablet_copy_client.cc:418] Wrote new consensus metadata
0807 01:47:43.692345 (+ 329us) tablet_replica.cc:206] Creating consensus instance
0807 01:47:43.692648 (+ 303us) ts_tablet_manager.cc:989] Replica registered.
0807 01:47:43.692714 (+ 66us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":56896,"tablet-copy.queue_time_us":82305,"thread_start_us":231623,"threads_started":2}
I20260807 01:47:43.797183 30673 tablet_copy_client.cc:806] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:43.805395 30673 tablet_copy_client.cc:670] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:43.774706 27170 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.803254 30787 consensus_queue.cc:260] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:43.808485 30787 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:43.797874 30725 raft_consensus.cc:740] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:43.809589 30731 raft_consensus.cc:2749] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 01:47:43.810120 30725 consensus_queue.cc:260] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:43.811343 30725 raft_consensus.cc:3060] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:43.812389 27170 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.812685 28706 raft_consensus.cc:1275] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEARNER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 01:47:43.813190 30731 raft_consensus.cc:2749] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 01:47:43.799612 30781 consensus_queue.cc:260] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:43.814730 27170 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.815448 30731 raft_consensus.cc:2749] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260807 01:47:43.815536 30781 raft_consensus.cc:3060] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:43.819175 30790 consensus_queue.cc:1059] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260807 01:47:43.804625 30739 tablet_copy_client.cc:806] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 0 data blocks...
I20260807 01:47:43.776261 30702 raft_consensus.cc:740] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:43.821491 30702 consensus_queue.cc:260] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:43.822979 30702 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:43.820421 30769 raft_consensus.cc:3055] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Stepping down as leader of term 4
W20260807 01:47:43.806707 30763 consensus_queue.cc:1175] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3478729505b046c2a0051c2d0130873f" responder_term: 4 status { last_received { term: 4 index: 4 } last_committed_idx: 3 last_received_current_leader { term: 4 index: 4 } }
I20260807 01:47:43.830749 30769 raft_consensus.cc:740] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:43.831651 30769 consensus_queue.cc:260] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:43.820468 30739 tablet_copy_client.cc:670] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Starting download of 1 WAL segments...
I20260807 01:47:43.832937 30769 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:43.798578 30694 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:43.807132 30779 raft_consensus.cc:2749] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 01:47:43.819680 30681 consensus_queue.cc:1048] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260807 01:47:43.864212 30738 tablet_copy_client.cc:538] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:43.869895 30673 tablet_copy_client.cc:538] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:43.882041 27287 leader_election.cc:304] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
W20260807 01:47:43.883239 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 98826us
I20260807 01:47:43.884528 27287 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:43.899631 27331 raft_consensus.cc:1275] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 01:47:43.899534 27064 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:43.903710 30681 consensus_queue.cc:1059] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:43.905269 30681 raft_consensus.cc:3055] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:43.906499 27063 leader_election.cc:304] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.907763 27063 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.909075 27063 leader_election.cc:304] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.887081 30790 raft_consensus.cc:2749] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 01:47:43.889508 30683 consensus_queue.cc:1059] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260807 01:47:43.910904 30681 raft_consensus.cc:740] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:43.888149 30779 raft_consensus.cc:2749] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 01:47:43.914757 30681 consensus_queue.cc:260] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:43.916270 30681 raft_consensus.cc:3060] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:43.921885 30673 tablet_bootstrap.cc:492] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:43.920898 26952 catalog_manager.cc:5697] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 4 to 5, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4). New cstate: current_term: 5 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:43.935528 30694 raft_consensus.cc:2749] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:43.935849 30800 raft_consensus.cc:2749] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:43.936314 30694 raft_consensus.cc:2749] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:43.936630 30800 raft_consensus.cc:2749] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:43.937827 27064 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:43.944336 30683 raft_consensus.cc:3055] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:47:43.944808 30683 raft_consensus.cc:740] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:43.953289 30683 consensus_queue.cc:260] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:43.954793 30683 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:43.966051 30800 raft_consensus.cc:2749] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:47:43.967156 30778 consensus_queue.cc:1048] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 01:47:43.968076 27064 leader_election.cc:304] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:43.969472 30778 raft_consensus.cc:2749] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 01:47:43.934973 30801 consensus_queue.cc:1059] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:43.981199 30783 consensus_queue.cc:1059] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260807 01:47:43.991655 30801 raft_consensus.cc:3055] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:43.991786 30696 raft_consensus.cc:3055] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 7 LEADER]: Stepping down as leader of term 7
I20260807 01:47:43.992151 30801 raft_consensus.cc:740] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:43.992403 30696 raft_consensus.cc:740] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:43.996773 27226 raft_consensus.cc:1275] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 01:47:44.001369 27105 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:41080
I20260807 01:47:44.002067 27105 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:47:44.002467 27105 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:43.987525 30738 tablet_bootstrap.cc:492] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:43.990069 30787 consensus_queue.cc:1059] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:43.993736 30696 consensus_queue.cc:260] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:44.012681 30696 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:47:43.997262 30801 consensus_queue.cc:260] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:44.007226 27109 raft_consensus.cc:1240] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:44.016907 30778 consensus_queue.cc:1048] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260807 01:47:44.032899 30801 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:44.050710 30739 tablet_copy_client.cc:538] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 01:47:44.052700 30731 raft_consensus.cc:3055] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:44.053210 30731 raft_consensus.cc:740] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:44.054188 30731 consensus_queue.cc:260] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:44.056531 30731 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:44.058619 27105 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.070546 27100 raft_consensus.cc:1240] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 4. Current term is 5. Ops: []
I20260807 01:47:44.074129 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:44.074662 30708 consensus_queue.cc:1059] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:47:44.075155 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:44.075911 28709 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:44.074659 27105 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:44.077528 27216 raft_consensus.cc:3055] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:44.078075 27216 raft_consensus.cc:740] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:44.078907 27216 consensus_queue.cc:260] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.079807 27216 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:44.087507 30801 raft_consensus.cc:2955] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Committing config change with OpId 5.6: config changed from index 5 to 6, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:44.089648 27216 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
W20260807 01:47:44.090369 30708 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260807 01:47:44.092470 27216 raft_consensus.cc:2955] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 5 to 6, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:44.093811 27064 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:44.095184 30778 raft_consensus.cc:2804] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:47:44.095675 30778 raft_consensus.cc:697] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:44.096524 30778 consensus_queue.cc:237] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.103627 28709 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:44.123883 30739 tablet_bootstrap.cc:492] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap starting.
I20260807 01:47:44.124178 26948 catalog_manager.cc:5697] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f reported cstate change: config changed from index 5 to 6, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 5 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:44.145979 30760 raft_consensus.cc:493] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:44.156250 30760 raft_consensus.cc:515] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.156013 28706 raft_consensus.cc:2955] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 5 to 6, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:44.158996 30760 leader_election.cc:290] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:44.161053 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:44.161845 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:44.164788 27331 raft_consensus.cc:2468] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:47:44.168656 28655 leader_election.cc:304] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:47:44.170786 30736 raft_consensus.cc:2804] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:47:44.171517 30736 raft_consensus.cc:493] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:44.172142 30736 raft_consensus.cc:3060] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:44.189375 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:44.191372 26954 catalog_manager.cc:5697] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 4 to 5, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 5 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:44.253764 27330 raft_consensus.cc:2955] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 5 to 6, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:44.263800 27109 raft_consensus.cc:1275] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 01:47:44.265885 30730 consensus_queue.cc:1048] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:47:44.276391 30736 raft_consensus.cc:515] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.280611 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:44.279796 30736 leader_election.cc:290] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:44.299959 30776 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:44.300730 30776 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.304857 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:44.305315 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:44.305755 28709 raft_consensus.cc:2393] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 8: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:47:44.306012 27227 raft_consensus.cc:2468] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 8.
I20260807 01:47:44.309422 27287 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:44.303970 30776 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:44.311872 30711 raft_consensus.cc:2804] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:47:44.312301 30711 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
W20260807 01:47:44.312507 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 98394.4us
I20260807 01:47:44.315376 27331 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:44.315908 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:44.317205 27337 raft_consensus.cc:3060] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
W20260807 01:47:44.322080 30738 log.cc:927] Time spent T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.064s user 0.004s sys 0.001s
I20260807 01:47:44.326537 30738 tablet_bootstrap.cc:492] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:44.327548 30738 tablet_bootstrap.cc:492] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:44.328415 30738 ts_tablet_manager.cc:1403] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.341s user 0.018s sys 0.021s
W20260807 01:47:44.312201 30809 log.cc:927] Time spent T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df: Append to log took a long time: real 0.060s user 0.001s sys 0.000s
I20260807 01:47:44.329025 30711 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:47:44.357240 27331 raft_consensus.cc:1275] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 01:47:44.367316 30711 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.341426 27337 raft_consensus.cc:2468] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:47:44.369161 30738 raft_consensus.cc:359] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:44.376106 28655 leader_election.cc:304] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:47:44.378860 30742 consensus_queue.cc:1048] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:47:44.381263 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:44.382797 30816 raft_consensus.cc:2804] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:44.383496 30816 raft_consensus.cc:697] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:44.384591 30816 consensus_queue.cc:237] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.389660 30738 raft_consensus.cc:740] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:44.395000 30738 consensus_queue.cc:260] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:44.408807 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:44.410529 28706 raft_consensus.cc:2393] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 8: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:47:44.419494 27227 raft_consensus.cc:2468] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 8.
I20260807 01:47:44.431365 30776 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:44.432629 27287 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:44.419198 30711 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:44.438108 30779 raft_consensus.cc:2804] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader election won for term 8
I20260807 01:47:44.460364 27227 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:59426
I20260807 01:47:44.461234 27227 raft_consensus.cc:606] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:44.461977 27227 raft_consensus.cc:614] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 01:47:44.432009 30776 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.452126 30738 ts_tablet_manager.cc:1434] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.123s user 0.005s sys 0.008s
I20260807 01:47:44.457165 30673 tablet_bootstrap.cc:492] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260807 01:47:44.469743 24592 auto_leader_rebalancer.cc:467] leader transfer for tablet fabde7ae28974f7380cb7641ab9fd6b1 (from 3478729505b046c2a0051c2d0130873f to 3262a9d183914b699227fdba0ca5bed3) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 01:47:44.484193 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:47:44.496033 30739 tablet_bootstrap.cc:492] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 01:47:44.497149 30739 tablet_bootstrap.cc:492] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:44.497903 30739 ts_tablet_manager.cc:1403] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.375s user 0.026s sys 0.014s
I20260807 01:47:44.500571 30776 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:44.502810 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:47:44.503679 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:44.503880 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:44.506287 27227 raft_consensus.cc:2468] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:47:44.465651 30779 raft_consensus.cc:697] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 8 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:44.507649 27287 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:44.517938 30673 tablet_bootstrap.cc:492] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12: Bootstrap complete.
I20260807 01:47:44.517544 30769 raft_consensus.cc:2804] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:47:44.516868 30737 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:44.525142 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:44.539831 28706 raft_consensus.cc:2393] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 2: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:47:44.546922 30779 consensus_queue.cc:237] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.542497 30737 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.549958 30737 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:44.552839 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:44.553486 27216 raft_consensus.cc:2468] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 8.
I20260807 01:47:44.556607 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:44.556377 28660 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:44.563736 30813 raft_consensus.cc:2804] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 01:47:44.563902 27250 tablet_copy_service.cc:342] P 3478729505b046c2a0051c2d0130873f: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-de3b20c9b4a7467ea9e6c098167df7e3 received from {username='slave'} at 127.0.0.1:41230
I20260807 01:47:44.534317 30769 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:44.564278 30813 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:44.570055 30813 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:47:44.570381 27250 tablet_copy_service.cc:434] P 3478729505b046c2a0051c2d0130873f: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-de3b20c9b4a7467ea9e6c098167df7e3 on tablet de3b20c9b4a7467ea9e6c098167df7e3 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:44.575946 27216 raft_consensus.cc:1275] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 01:47:44.576685 30769 raft_consensus.cc:3060] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:44.580214 30781 consensus_queue.cc:1048] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:47:44.566784 30673 ts_tablet_manager.cc:1403] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12: Time spent bootstrapping tablet: real 0.645s user 0.033s sys 0.013s
I20260807 01:47:44.587839 27109 consensus_queue.cc:237] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.589885 30673 raft_consensus.cc:359] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:44.590878 30673 raft_consensus.cc:740] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:44.593775 30739 raft_consensus.cc:359] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:44.594693 30739 raft_consensus.cc:740] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Initialized, Role: LEARNER
I20260807 01:47:44.595408 30739 consensus_queue.cc:260] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:44.591578 30673 consensus_queue.cc:260] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:44.630120 30789 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:44.632006 30739 ts_tablet_manager.cc:1434] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.123s user 0.003s sys 0.006s
I20260807 01:47:44.629232 26952 catalog_manager.cc:5697] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:44.630774 30789 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:44.639279 30673 ts_tablet_manager.cc:1434] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12: Time spent starting tablet: real 0.054s user 0.008s sys 0.001s
I20260807 01:47:44.642581 30736 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:44.621848 30813 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.650364 27250 tablet_copy_service.cc:342] P 3478729505b046c2a0051c2d0130873f: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-00395257beb442e892a4ec18c4c30d95 received from {username='slave'} at 127.0.0.1:41230
I20260807 01:47:44.650872 27250 tablet_copy_service.cc:434] P 3478729505b046c2a0051c2d0130873f: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-00395257beb442e892a4ec18c4c30d95 on tablet 00395257beb442e892a4ec18c4c30d95 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:44.643391 30736 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.652647 27126 tablet_copy_service.cc:342] P 3262a9d183914b699227fdba0ca5bed3: Request end of tablet copy session 57bb4985157447f0a3ab680fa0026f12-67fed460b08e4124a6d4484db52478b1 received from {username='slave'} at 127.0.0.1:52556
I20260807 01:47:44.653415 27126 tablet_copy_service.cc:434] P 3262a9d183914b699227fdba0ca5bed3: ending tablet copy session 57bb4985157447f0a3ab680fa0026f12-67fed460b08e4124a6d4484db52478b1 on tablet 67fed460b08e4124a6d4484db52478b1 with peer 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:44.653815 30736 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:44.646395 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:44.642405 30789 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:44.654287 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:44.655225 27216 raft_consensus.cc:2393] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 6: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:44.655154 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:44.653738 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:44.655952 27331 raft_consensus.cc:2468] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 2.
I20260807 01:47:44.656179 27227 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:47:44.648591 30769 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.658519 30736 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:44.659322 30813 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:44.659119 30736 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.659991 28655 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:44.661393 30736 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:44.662061 30736 raft_consensus.cc:2804] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:47:44.662469 30736 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:44.662871 30736 raft_consensus.cc:3060] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:44.663714 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:44.664759 27331 raft_consensus.cc:2468] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
I20260807 01:47:44.668515 30711 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:44.669209 30711 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.671566 30711 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:44.673517 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:44.674501 28702 raft_consensus.cc:2393] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 3: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:47:44.676906 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:44.677829 28706 raft_consensus.cc:2393] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 2: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:47:44.669234 28655 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:44.649720 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:44.685211 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:44.727188 27216 raft_consensus.cc:2393] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 2: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:44.685294 27337 raft_consensus.cc:2468] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:47:44.685592 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:44.729411 27226 raft_consensus.cc:2393] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 7: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:44.689827 28706 raft_consensus.cc:1275] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 01:47:44.699460 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:44.732501 27100 raft_consensus.cc:2468] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:47:44.694806 30769 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:44.681706 30737 raft_consensus.cc:2804] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:47:44.734189 30737 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:44.734652 30737 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:44.737411 30681 consensus_queue.cc:1048] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 01:47:44.686962 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:44.714896 28702 raft_consensus.cc:1275] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 01:47:44.743925 27331 raft_consensus.cc:1275] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 01:47:44.733526 27063 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:47:44.745301 27287 leader_election.cc:304] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:44.703088 30736 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.746099 30801 consensus_queue.cc:1048] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 01:47:44.748026 30736 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:44.745858 30789 raft_consensus.cc:2804] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:47:44.748981 30711 raft_consensus.cc:2804] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:44.749500 30711 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:44.749945 30711 raft_consensus.cc:3060] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:44.752118 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:44.748198 30801 consensus_queue.cc:1048] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260807 01:47:44.680164 27227 raft_consensus.cc:2468] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 9.
I20260807 01:47:44.759039 30711 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.749006 30789 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:44.760322 30789 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:44.762403 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:44.763267 28706 raft_consensus.cc:2393] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 3: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:47:44.765317 30696 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:44.766000 30696 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:44.768188 30696 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:44.771662 30711 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:44.770944 30781 raft_consensus.cc:2955] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Committing config change with OpId 5.7: config changed from index 6 to 7, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:44.776527 27331 raft_consensus.cc:2955] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Committing config change with OpId 5.7: config changed from index 6 to 7, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:44.778528 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:44.784929 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:44.785813 27326 raft_consensus.cc:2393] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 6: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:44.783738 28660 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:47:44.790269 30813 raft_consensus.cc:2804] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 01:47:44.791087 30813 raft_consensus.cc:697] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 9 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:44.792052 30813 consensus_queue.cc:237] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.795585 27213 raft_consensus.cc:2468] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:47:44.796999 27287 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:44.800704 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:44.802402 30711 raft_consensus.cc:2804] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:47:44.803539 28706 raft_consensus.cc:2955] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Committing config change with OpId 5.7: config changed from index 6 to 7, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:47:44.805812 27100 raft_consensus.cc:2468] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:47:44.802652 27227 raft_consensus.cc:2393] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 2: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:44.810007 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:44.810774 27101 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:47:44.809937 30711 raft_consensus.cc:697] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:44.815506 30711 consensus_queue.cc:237] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.818648 27172 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:47:44.819954 30696 raft_consensus.cc:2804] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:47:44.820421 30696 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:44.820851 30696 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:44.821185 30789 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:44.824805 27337 raft_consensus.cc:2468] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 2.
I20260807 01:47:44.828359 28655 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:47:44.829649 30736 raft_consensus.cc:2804] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 01:47:44.830680 30736 raft_consensus.cc:697] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:44.831809 30736 consensus_queue.cc:237] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.839416 26932 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1c4e145f8f52431fa41b1f596e2f1133 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 01:47:44.851045 27287 leader_election.cc:304] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:44.852056 30769 raft_consensus.cc:2804] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:47:44.852630 30769 raft_consensus.cc:697] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:44.853614 30769 consensus_queue.cc:237] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.866724 30708 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:44.867451 30708 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:44.869690 30708 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:44.879769 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:44.880761 27337 raft_consensus.cc:2393] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 5: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:44.882355 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:44.883916 27227 raft_consensus.cc:2393] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 6: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:44.888866 26952 catalog_manager.cc:5697] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index 6 to 7, VOTER 3478729505b046c2a0051c2d0130873f (127.24.4.2) evicted. New cstate: current_term: 5 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:44.891067 30696 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:44.891680 30737 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.894553 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:44.895269 27101 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:47:44.895362 30789 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:44.897444 30737 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:44.898226 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:44.897598 26950 catalog_manager.cc:5697] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 7 to 8, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 8 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:44.898545 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:44.904001 27172 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:47:44.905223 30708 raft_consensus.cc:2804] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:47:44.905620 30708 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:44.905987 30708 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:44.915961 27199 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c4e145f8f52431fa41b1f596e2f1133 with delete_type TABLET_DATA_TOMBSTONED (TS 3478729505b046c2a0051c2d0130873f not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:53780
I20260807 01:47:44.920835 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:44.924799 27227 raft_consensus.cc:2393] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 7: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:44.921576 27326 raft_consensus.cc:2468] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
I20260807 01:47:44.930972 28655 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:44.932626 30736 raft_consensus.cc:2804] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:47:44.933156 30736 raft_consensus.cc:697] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 7 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:44.936383 27326 raft_consensus.cc:1275] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 01:47:44.921491 30708 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:44.942126 30813 consensus_queue.cc:1048] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:47:44.943332 26950 catalog_manager.cc:5697] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 8 to 9, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4). New cstate: current_term: 9 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:44.952168 27337 raft_consensus.cc:2468] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:47:44.953488 27063 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:47:44.956149 26952 catalog_manager.cc:5697] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 1 to 2, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 2 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:44.957957 26952 catalog_manager.cc:5697] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 2 to 3, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 3 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:44.959779 30789 raft_consensus.cc:2804] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:47:44.934921 30736 consensus_queue.cc:237] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:44.962311 30789 raft_consensus.cc:697] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:44.970618 30789 consensus_queue.cc:237] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:44.977247 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:44.985536 30708 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:44.996556 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:44.996978 27227 raft_consensus.cc:3060] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 4
I20260807 01:47:44.992451 27101 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:47:44.999038 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:44.999469 27172 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:44.999665 30769 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.000195 30769 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:45.003036 30769 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:45.004542 30696 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.004799 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:45.005475 30696 raft_consensus.cc:2804] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:47:45.006945 30696 raft_consensus.cc:697] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 6 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:45.011312 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:45.012266 27331 raft_consensus.cc:2393] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 6: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:45.015560 27109 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:47:45.012080 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:45.016497 27100 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:47:45.018054 27287 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:45.022775 27172 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:45.026532 27337 raft_consensus.cc:2393] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 5: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:45.037885 30779 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.037547 30708 raft_consensus.cc:2804] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:47:45.046217 30779 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.046339 30839 tablet_replica.cc:333] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f: stopping tablet replica
I20260807 01:47:45.034922 30711 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.048437 27227 raft_consensus.cc:1275] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 01:47:45.034970 30769 raft_consensus.cc:2804] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:47:45.011327 30696 consensus_queue.cc:237] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.046885 30708 raft_consensus.cc:697] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:45.062439 30769 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:45.073133 30769 raft_consensus.cc:3060] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:45.072285 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:45.074268 27102 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:47:45.076051 26954 catalog_manager.cc:5697] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 5 to 6, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 6 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:45.052172 30711 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:45.080322 30711 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:45.066341 30742 consensus_queue.cc:1048] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260807 01:47:45.081470 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:45.082484 30839 raft_consensus.cc:2243] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Raft consensus shutting down.
I20260807 01:47:45.082891 27287 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:45.083444 30839 raft_consensus.cc:2272] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Raft consensus is shut down!
I20260807 01:47:45.072220 30779 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:45.084012 30769 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:45.085076 30779 raft_consensus.cc:2804] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:47:45.085492 30779 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:45.085873 30779 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:45.088871 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:45.091077 30769 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:45.091748 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:45.092418 27101 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 7.
I20260807 01:47:45.093253 30708 consensus_queue.cc:237] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.094340 28702 raft_consensus.cc:1240] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [term 9 LEADER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 8. Current term is 9. Ops: []
I20260807 01:47:45.095533 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:45.096221 27213 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:47:45.105376 30776 consensus_queue.cc:1059] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260807 01:47:45.106336 30839 ts_tablet_manager.cc:1916] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 01:47:45.106271 30776 consensus_queue.cc:1243] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer 57bb4985157447f0a3ab680fa0026f12 log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260807 01:47:45.128921 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 46118.1us
I20260807 01:47:45.131247 27287 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:45.131583 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:45.132205 27101 raft_consensus.cc:3060] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:45.133005 30787 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.138345 30787 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.146920 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:45.173976 27102 raft_consensus.cc:2468] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:47:45.176644 26952 catalog_manager.cc:5697] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 1 to 2, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4). New cstate: current_term: 2 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:45.177784 27101 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:47:45.179188 26952 catalog_manager.cc:5697] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 6 to 7, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4). New cstate: current_term: 7 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:45.175981 27172 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:45.111337 30779 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:45.200316 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 59481.6us
I20260807 01:47:45.134974 30846 raft_consensus.cc:2804] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:47:45.173036 30816 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.172852 30787 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.198005 30708 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.223152 30783 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.201150 30813 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:45.198596 30778 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:45.234653 30816 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:45.218855 30846 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:45.285300 30854 raft_consensus.cc:2804] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:45.285961 30854 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:45.286363 30854 raft_consensus.cc:3060] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:45.287566 30851 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:45.288251 30851 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.281452 30846 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Advancing to term 8
W20260807 01:47:45.302059 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 126165us
I20260807 01:47:45.308235 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:45.309109 27337 raft_consensus.cc:2468] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:47:45.308166 27287 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:45.256222 30813 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:45.279901 30742 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:45.290112 30787 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:45.258535 30778 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.251034 30708 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:45.318341 30711 raft_consensus.cc:3055] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 8 LEADER]: Stepping down as leader of term 8
I20260807 01:47:45.254000 30783 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:45.356727 30849 log.cc:927] Time spent T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.099s user 0.001s sys 0.000s
I20260807 01:47:45.374058 30737 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.374935 30737 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:45.351601 30841 log.cc:927] Time spent T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.214s user 0.001s sys 0.000s
I20260807 01:47:45.340287 30787 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:45.355436 30711 raft_consensus.cc:740] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:45.336787 30742 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:45.393726 30769 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:45.392365 30856 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
W20260807 01:47:45.400180 30845 log.cc:927] Time spent T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.189s user 0.001s sys 0.000s
I20260807 01:47:45.395774 30867 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.399142 30778 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.408794 30779 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:45.397156 30865 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:45.401408 30813 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.403502 30816 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:45.402522 30870 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.399462 30857 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
W20260807 01:47:45.489876 30852 log.cc:927] Time spent T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.132s user 0.001s sys 0.000s
I20260807 01:47:45.404505 30853 raft_consensus.cc:2804] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:45.495635 30853 raft_consensus.cc:697] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:45.496665 30853 consensus_queue.cc:237] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:45.483130 30856 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.493495 30857 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:45.486816 30867 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.500917 30867 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:47:45.536759 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 228445us
W20260807 01:47:45.452049 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 131380us
I20260807 01:47:45.567695 30708 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.569257 30708 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.571439 30708 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:45.572638 30696 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.573666 30696 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:45.575951 30708 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.580751 30708 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.576969 30696 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:45.584993 30742 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.585850 30776 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:45.483309 30769 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.587462 30779 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:45.584995 30737 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.589684 30778 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.592103 30781 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.594734 30783 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.606447 30776 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.617405 30776 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:45.492357 30870 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:45.581768 30711 consensus_queue.cc:260] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:45.622877 30863 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.623669 30778 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:45.624262 30863 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.627120 30779 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.630487 30857 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.633035 30769 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:45.634519 30870 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:45.637253 30856 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.639637 30779 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:45.639464 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:45.499042 30865 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.641289 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:45.641913 30787 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:45.624818 30711 raft_consensus.cc:3060] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:47:45.680454 27227 raft_consensus.cc:2468] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:47:45.648159 30851 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.650030 30813 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.653395 28706 raft_consensus.cc:1275] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260807 01:47:45.630182 30781 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.655230 30854 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.660120 30778 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.662793 27216 raft_consensus.cc:1275] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:45.664678 27222 raft_consensus.cc:1275] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260807 01:47:45.665444 27213 raft_consensus.cc:1275] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260807 01:47:45.671222 30863 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:45.701443 30813 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:45.704514 30813 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.707962 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:45.709221 27218 raft_consensus.cc:2393] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 5: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:45.711886 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:45.712749 27229 raft_consensus.cc:2468] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:47:45.714321 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:45.715554 27101 raft_consensus.cc:2368] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for earlier term 3. Current term is 4.
I20260807 01:47:45.715487 30891 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.716419 30883 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.716573 30891 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:45.717099 30883 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:45.678141 30801 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.697428 27226 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 2.
I20260807 01:47:45.690589 30708 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.725409 30708 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.726171 30787 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.726873 30787 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:45.728976 30776 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:45.730762 30787 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:45.718870 30859 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.729784 30901 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.737964 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:45.737995 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:45.739214 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:45.740718 30854 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.744848 27213 raft_consensus.cc:2468] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 2.
I20260807 01:47:45.750536 27105 raft_consensus.cc:2393] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 7: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:47:45.751093 30901 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:45.750620 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:45.751487 30859 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:45.752642 27101 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:47:45.757378 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:45.782483 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:45.765486 30883 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.785022 27099 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:47:45.764302 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:45.755259 30865 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:45.762449 30801 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.761782 27102 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 7
I20260807 01:47:45.731283 30776 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:45.787832 30865 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.790405 30865 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.792940 30865 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:45.786736 27337 raft_consensus.cc:2468] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:47:45.751587 27230 raft_consensus.cc:1275] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260807 01:47:45.797896 30888 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.798811 30888 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.799983 27330 raft_consensus.cc:1275] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 01:47:45.799856 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:45.799767 27336 raft_consensus.cc:1275] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260807 01:47:45.801556 28702 raft_consensus.cc:2468] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:47:45.778138 30903 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.761144 30708 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.802671 30903 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.803452 27289 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:45.759894 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:45.803690 30708 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.759269 30859 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.800362 30779 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:45.754022 30901 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:45.814836 30779 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:45.817277 30779 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:45.817857 30779 raft_consensus.cc:2804] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:47:45.819022 30779 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:45.819559 30779 raft_consensus.cc:3060] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:45.757447 30781 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.769626 30891 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.824175 30708 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.802671 30896 consensus_queue.cc:1048] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260807 01:47:45.827917 30776 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:45.801461 30888 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:45.828462 30801 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.837445 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:45.843829 27099 raft_consensus.cc:2393] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 3: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:47:45.847210 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:45.847926 27336 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:45.846261 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:45.858229 27230 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:45.864298 28660 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
W20260807 01:47:45.865659 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 122091us
I20260807 01:47:45.865815 30883 raft_consensus.cc:2804] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:47:45.866446 30883 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:45.866860 30883 raft_consensus.cc:3060] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:45.786738 30846 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:45.875411 30813 consensus_queue.cc:1048] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260807 01:47:45.875957 30879 consensus_queue.cc:1048] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:45.876405 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:45.878916 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:45.879703 27337 raft_consensus.cc:1275] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260807 01:47:45.880215 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:45.880846 27327 raft_consensus.cc:2468] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:47:45.881186 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:45.878958 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:45.881999 27338 raft_consensus.cc:2468] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:47:45.882654 27336 raft_consensus.cc:2468] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:47:45.884330 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:45.885020 27338 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:45.885342 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:45.887015 27336 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:45.879642 27326 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:47:45.889321 27331 raft_consensus.cc:2368] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for earlier term 7. Current term is 8.
I20260807 01:47:45.889263 27099 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:47:45.891801 27330 raft_consensus.cc:1275] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 10. (term mismatch)
I20260807 01:47:45.893559 27330 pending_rounds.cc:85] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade: Aborting all ops after (but not including) 9
I20260807 01:47:45.893664 28660 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:45.893957 27330 pending_rounds.cc:107] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade: Aborting uncommitted NO_OP operation due to leader change: 8.10
I20260807 01:47:45.894608 27330 raft_consensus.cc:2889] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 01:47:45.894861 30813 raft_consensus.cc:2804] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:47:45.895438 30813 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:45.895840 30813 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:45.892349 30903 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.891163 30846 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:45.892899 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:45.898943 28660 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:45.899829 28706 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Advancing to term 8
I20260807 01:47:45.899931 30742 raft_consensus.cc:2804] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:45.901116 30742 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:45.901526 30742 raft_consensus.cc:3060] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:45.899080 30879 consensus_queue.cc:1048] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260807 01:47:45.907545 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:45.910012 30742 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:45.913575 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:45.914706 27100 raft_consensus.cc:2393] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 4: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:47:45.914726 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:45.916754 27105 raft_consensus.cc:2393] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 2: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:47:45.915740 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:45.917925 27109 raft_consensus.cc:2393] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 3: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:47:45.919252 27101 raft_consensus.cc:1275] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260807 01:47:45.919680 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:45.920771 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:45.924357 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:45.925194 27105 raft_consensus.cc:2468] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:47:45.926403 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:45.927155 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:45.927724 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:45.928422 27101 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:47:45.927889 27104 raft_consensus.cc:3060] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:45.929145 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:45.930205 27108 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8.
I20260807 01:47:45.934648 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:45.927102 27100 raft_consensus.cc:2468] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:45.938742 30736 consensus_queue.cc:1048] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:45.940506 27105 raft_consensus.cc:1275] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260807 01:47:45.941324 30779 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:45.945046 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:45.945737 30839 ts_tablet_manager.cc:1929] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 5.6
I20260807 01:47:45.946508 28655 leader_election.cc:400] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Vote denied by peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429) with higher term. Message: Invalid argument: T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for earlier term 7. Current term is 8.
I20260807 01:47:45.946970 28655 leader_election.cc:403] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260807 01:47:45.946501 27105 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:47:45.938728 27109 raft_consensus.cc:2468] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:47:45.936463 27102 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 7.
I20260807 01:47:45.937187 30886 consensus_queue.cc:1048] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260807 01:47:45.968547 30878 consensus_queue.cc:1048] T 08edf043c99f445b99e723211e8e0ad0 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260807 01:47:45.970727 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:45.961073 30846 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.972376 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:45.974323 27104 raft_consensus.cc:2468] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:47:45.946538 30839 log.cc:1199] T 1c4e145f8f52431fa41b1f596e2f1133 P 3478729505b046c2a0051c2d0130873f: Deleting WAL directory at /tmp/dist-test-taskEGVQhX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786067112798834-24592-0/minicluster-data/ts-1-root/wals/1c4e145f8f52431fa41b1f596e2f1133
I20260807 01:47:45.982630 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:45.983644 30846 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.982729 30854 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.987574 30854 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.987881 27063 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:45.989912 30891 raft_consensus.cc:2804] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:47:45.990341 30891 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:45.990686 30891 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:45.991127 30854 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.982887 30711 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:45.991647 30813 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:45.992300 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:45.992144 30711 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:45.992656 30742 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:45.992899 27213 raft_consensus.cc:3060] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:45.994452 30711 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:45.996331 30813 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:45.985440 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
W20260807 01:47:46.000475 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:46.000682 28707 raft_consensus.cc:2368] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 3. Current term is 4.
I20260807 01:47:46.005762 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:46.014528 30846 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:46.019850 28709 raft_consensus.cc:3060] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Advancing to term 4
I20260807 01:47:46.018463 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:46.022691 27226 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 4
I20260807 01:47:46.001178 26950 catalog_manager.cc:5697] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 5 to 6, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 6 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:46.031693 27063 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:46.048291 27063 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:46.056223 30781 raft_consensus.cc:2804] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:47:46.074416 27213 raft_consensus.cc:2468] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:47:46.077371 27226 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:47:46.070987 30781 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:46.085852 27063 leader_election.cc:304] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:46.040189 30801 consensus_queue.cc:1048] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260807 01:47:46.083016 28660 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:47:46.087406 28660 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:46.088974 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:46.091059 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:46.091686 27336 raft_consensus.cc:2468] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:47:46.092587 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:46.096527 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:46.097410 27328 raft_consensus.cc:2468] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:47:46.098088 30900 raft_consensus.cc:2804] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:46.098584 30900 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:46.098927 30900 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:46.102316 28702 raft_consensus.cc:1240] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:46.093117 27327 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:46.107065 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:47:46.113636 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 44903.4us
I20260807 01:47:46.089709 30781 raft_consensus.cc:3060] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:46.116849 27063 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:46.090723 30742 raft_consensus.cc:2804] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:46.124387 27063 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:46.079340 30883 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:46.126832 27105 raft_consensus.cc:1275] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260807 01:47:46.081651 30846 consensus_queue.cc:1048] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260807 01:47:46.125085 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:46.129566 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:46.109850 30886 raft_consensus.cc:2804] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:47:46.129720 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:46.133514 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:46.137058 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:46.138017 27327 raft_consensus.cc:2393] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 3: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:46.139613 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:46.134832 28660 leader_election.cc:400] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for earlier term 3. Current term is 4.
I20260807 01:47:46.140332 27226 raft_consensus.cc:2468] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:46.140427 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:46.141427 27331 raft_consensus.cc:2393] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 4: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:46.140726 28660 leader_election.cc:403] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260807 01:47:46.139515 27213 raft_consensus.cc:2468] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:46.129645 30859 raft_consensus.cc:2804] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:47:46.132154 30801 raft_consensus.cc:2804] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:47:46.147565 27099 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:47:46.147055 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:46.148478 27328 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:47:46.130635 28706 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 8.
I20260807 01:47:46.114472 30896 consensus_queue.cc:1059] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260807 01:47:46.147989 26948 catalog_manager.cc:5697] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:46.150399 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:46.154489 27216 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:47:46.152562 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:46.149487 30801 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:46.142493 30886 raft_consensus.cc:697] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:46.146989 30742 raft_consensus.cc:697] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:46.163708 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:46.153661 30856 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:46.151675 30859 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:46.151393 30779 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:46.166497 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:46.156164 30902 raft_consensus.cc:2804] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:46.163971 28713 raft_consensus.cc:2393] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 4: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:47:46.168516 27222 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 1.
I20260807 01:47:46.174146 27100 raft_consensus.cc:2393] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 8: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:47:46.170966 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:46.173307 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:46.170006 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:46.174989 27337 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:47:46.179150 27289 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:46.165999 30801 raft_consensus.cc:3060] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:46.170974 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:46.182293 27103 raft_consensus.cc:2393] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 5: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:47:46.182717 27216 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:46.184158 30742 consensus_queue.cc:237] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:46.167862 30859 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:46.178593 30839 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53780 (request call id 83) took 1258 ms. Trace:
I20260807 01:47:46.181679 27108 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:47:46.172748 30902 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:46.179251 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:46.192703 30875 raft_consensus.cc:2804] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader election won for term 8
I20260807 01:47:46.192378 30886 consensus_queue.cc:237] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:46.193141 30916 log.cc:927] Time spent T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.080s user 0.000s sys 0.001s
I20260807 01:47:46.193485 30860 raft_consensus.cc:2749] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429) with higher term. Message: Invalid argument: T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for earlier term 7. Current term is 8.
W20260807 01:47:46.198805 30925 log.cc:927] Time spent T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.071s user 0.001s sys 0.000s
W20260807 01:47:46.200212 30919 log.cc:927] Time spent T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.078s user 0.000s sys 0.001s
I20260807 01:47:46.189218 30839 rpcz_store.cc:276] 0807 01:47:44.915624 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:44.915766 (+ 142us) service_pool.cc:224] Handling call
0807 01:47:45.045895 (+130129us) ts_tablet_manager.cc:1151] Deleting tablet 1c4e145f8f52431fa41b1f596e2f1133
0807 01:47:45.045960 (+ 65us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0807 01:47:45.708588 (+662628us) tablet_metadata.cc:665] Metadata flushed
0807 01:47:45.945439 (+236851us) tablet_metadata.cc:665] Metadata flushed
0807 01:47:46.173964 (+228525us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":129300,"thread_start_us":127179,"threads_started":1}
I20260807 01:47:46.208515 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:46.211150 27333 raft_consensus.cc:1275] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260807 01:47:46.212726 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:46.218083 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:46.223228 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:46.213411 27216 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 7.
I20260807 01:47:46.213689 27172 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:46.237002 28709 raft_consensus.cc:1275] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 01:47:46.215996 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:46.237112 27063 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
W20260807 01:47:46.245481 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 75958.5us
I20260807 01:47:46.249874 27337 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:47:46.229640 30902 raft_consensus.cc:3060] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:46.248409 27328 raft_consensus.cc:2468] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:47:46.255853 27325 raft_consensus.cc:2393] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 5: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:46.259069 27336 raft_consensus.cc:2393] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 2: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:46.253044 27287 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:46.259919 27331 raft_consensus.cc:3060] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 3
I20260807 01:47:46.255203 30776 consensus_queue.cc:1048] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260807 01:47:46.260272 28702 raft_consensus.cc:2393] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 4: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:47:46.266470 27218 raft_consensus.cc:1240] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 8. Current term is 9. Ops: []
I20260807 01:47:46.267469 27170 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:46.268399 30851 raft_consensus.cc:2804] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:46.268882 30851 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:46.269274 30851 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:46.268638 27170 leader_election.cc:304] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:46.274036 27287 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:46.245141 30708 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:46.278765 27170 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
W20260807 01:47:46.275532 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 47963.9us
I20260807 01:47:46.247382 30875 raft_consensus.cc:697] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 8 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:46.285069 30787 raft_consensus.cc:2804] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:47:46.285640 30787 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:46.285998 30787 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:46.287186 30892 raft_consensus.cc:2749] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:47:46.288399 30883 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:46.294957 30875 consensus_queue.cc:237] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:46.302292 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:46.304877 30781 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:46.305527 30783 raft_consensus.cc:2804] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:47:46.306061 30783 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:46.306847 30783 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:46.228456 30856 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:46.257731 30736 raft_consensus.cc:2749] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for earlier term 3. Current term is 4.
I20260807 01:47:46.320124 27105 raft_consensus.cc:2393] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 4: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:47:46.314460 27170 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:46.276970 30779 raft_consensus.cc:2804] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260807 01:47:46.324118 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 63971us
I20260807 01:47:46.325191 27170 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
W20260807 01:47:46.326924 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 52972us
I20260807 01:47:46.327127 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:46.324214 30854 raft_consensus.cc:2749] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260807 01:47:46.337278 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 183225us
I20260807 01:47:46.326125 30779 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:46.329599 30893 raft_consensus.cc:2804] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 01:47:46.355691 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:46.357925 27213 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:47:46.355531 27172 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
W20260807 01:47:46.370640 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 44516us
I20260807 01:47:46.355500 30893 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:46.353566 30779 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:46.384087 27172 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:46.371942 27170 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:46.363325 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:46.375171 30854 raft_consensus.cc:2804] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:47:46.375687 30893 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:47:46.333046 30708 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:46.387956 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:46.352854 30776 raft_consensus.cc:2804] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:47:46.394935 27212 raft_consensus.cc:2368] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3.
I20260807 01:47:46.395888 27213 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:47:46.371587 30937 raft_consensus.cc:2804] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:47:46.388813 30854 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
W20260807 01:47:46.425444 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 53623.8us
I20260807 01:47:46.429299 27170 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:46.418185 30942 raft_consensus.cc:2804] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:47:46.443874 27170 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:46.423071 30892 raft_consensus.cc:2804] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:47:46.423316 30937 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:46.432633 30854 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:46.425504 30776 raft_consensus.cc:697] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 7 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:46.464839 30892 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:46.477392 30892 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:46.478627 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:46.479130 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:46.479686 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:46.479820 27226 raft_consensus.cc:2468] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:47:46.480706 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:46.480894 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:46.467012 30949 consensus_queue.cc:1048] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:46.482019 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:46.483127 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:46.483314 30949 raft_consensus.cc:2804] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:47:46.483680 27213 raft_consensus.cc:3060] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:46.483719 30949 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:46.478534 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:46.484683 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:46.474089 27212 raft_consensus.cc:1240] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:46.476385 30776 consensus_queue.cc:237] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:46.476263 30926 log.cc:927] Time spent T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.115s user 0.000s sys 0.001s
I20260807 01:47:46.492965 30943 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
W20260807 01:47:46.493496 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 90681.2us
I20260807 01:47:46.464897 30781 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:46.498872 27213 raft_consensus.cc:2468] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:47:46.474043 30937 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:46.459355 30942 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:46.502899 27287 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:46.490831 30813 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:46.527545 27287 leader_election.cc:400] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3.
I20260807 01:47:46.503944 30949 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:46.530745 30888 raft_consensus.cc:2804] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:46.526847 30942 raft_consensus.cc:3060] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
W20260807 01:47:46.534873 30930 log.cc:927] Time spent T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.109s user 0.000s sys 0.000s
W20260807 01:47:46.535811 30933 log.cc:927] Time spent T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.097s user 0.000s sys 0.000s
I20260807 01:47:46.502031 30801 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:46.530700 27287 leader_election.cc:403] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260807 01:47:46.530134 30943 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:46.522717 30856 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:47:46.566349 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 122562us
W20260807 01:47:46.570259 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 44942.8us
I20260807 01:47:46.542526 30888 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:46.536134 30813 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:46.594830 30893 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:46.619629 30813 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:46.620085 27230 raft_consensus.cc:2368] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 8. Current term is 9.
W20260807 01:47:46.621063 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 64628.6us
I20260807 01:47:46.623999 30888 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:46.633097 30893 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:46.575680 30856 raft_consensus.cc:2804] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:47:46.597491 27287 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:46.617116 30851 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
W20260807 01:47:46.642026 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 210221us
W20260807 01:47:46.642462 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 65054.1us
W20260807 01:47:46.641544 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 268852us
I20260807 01:47:46.664595 27227 raft_consensus.cc:2368] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3.
I20260807 01:47:46.598800 30708 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:46.611526 30801 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:46.641832 30856 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:46.613889 30859 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:46.645038 30865 raft_consensus.cc:2804] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:47:46.676515 30851 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:46.677085 30708 consensus_queue.cc:1048] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 01:47:46.609911 30783 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:46.678987 30891 consensus_meta.cc:287] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.688s user 0.012s sys 0.003s
I20260807 01:47:46.680794 30902 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:46.687500 30943 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:46.688988 30856 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:46.692005 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:46.709357 27099 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:47:46.672945 27172 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:47:46.642566 30779 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:46.712194 27229 raft_consensus.cc:2368] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5.
I20260807 01:47:46.700776 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:46.700778 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:47:46.705874 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 272770us
I20260807 01:47:46.693647 30865 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:46.713395 27108 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:47:46.691520 27331 raft_consensus.cc:2468] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:47:46.729892 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:46.730774 27213 raft_consensus.cc:2393] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 4: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:46.698554 30938 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:46.730140 27103 raft_consensus.cc:2410] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260807 01:47:46.716480 30854 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:46.739409 30938 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:46.718233 30865 raft_consensus.cc:3060] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
W20260807 01:47:46.750068 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 77095.3us
I20260807 01:47:46.772051 27172 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:46.726828 30903 raft_consensus.cc:2804] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:46.706847 30891 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:46.725709 30875 raft_consensus.cc:2749] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3.
W20260807 01:47:46.756492 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 254484us
I20260807 01:47:46.776329 30903 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:46.782850 30903 raft_consensus.cc:3060] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:46.788707 27226 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:47:46.798839 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:46.755223 30901 raft_consensus.cc:3055] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:46.813679 27099 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 8.
I20260807 01:47:46.806474 27109 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:47:46.784946 30957 raft_consensus.cc:2804] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:47:46.808054 30779 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:46.808795 30902 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:46.807786 30891 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:46.817999 30901 raft_consensus.cc:740] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:46.846958 27175 leader_election.cc:400] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Vote denied by peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225) with higher term. Message: Invalid argument: T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 3. Current term is 4.
I20260807 01:47:46.847390 27175 leader_election.cc:403] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260807 01:47:46.854918 30957 raft_consensus.cc:697] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:46.856127 30871 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:46.856814 30871 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:46.858922 30871 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:47:46.856173 30957 consensus_queue.cc:237] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:46.862622 30944 consensus_queue.cc:1048] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260807 01:47:46.864892 30900 consensus_meta.cc:287] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.766s user 0.010s sys 0.008s
W20260807 01:47:46.872112 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:47:46.875701 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 156946us
W20260807 01:47:46.875972 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 53089.7us
I20260807 01:47:46.880527 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:46.876626 27175 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:46.885900 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:46.878914 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:46.887501 27105 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:46.806052 30937 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:46.904847 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:46.877113 30851 raft_consensus.cc:2749] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225) with higher term. Message: Invalid argument: T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 3. Current term is 4.
I20260807 01:47:46.909224 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:46.831912 30903 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:46.861812 30949 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:46.864691 26932 catalog_manager.cc:5028] TS 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855): tablet 1c4e145f8f52431fa41b1f596e2f1133 (table test-workload [id=17e6f3a2915e4a39abd5a81e3ac64734]) successfully deleted
W20260807 01:47:46.961313 26932 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 96780.7us
I20260807 01:47:46.865785 30900 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:46.881014 30942 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:46.847255 30901 consensus_queue.cc:260] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:46.905879 30783 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:46.941082 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:46.928517 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:46.977305 27230 raft_consensus.cc:3060] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 5
I20260807 01:47:46.946871 28702 raft_consensus.cc:1275] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260807 01:47:46.940327 30892 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:46.950596 27336 raft_consensus.cc:2393] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 4: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:46.952054 27105 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:47:46.953438 30937 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:46.953879 26952 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60866 (request call id 215) took 1826 ms. Trace:
W20260807 01:47:46.887017 30787 consensus_meta.cc:287] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.601s user 0.012s sys 0.005s
I20260807 01:47:46.911224 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:47:46.930112 30968 log.cc:927] Time spent T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.085s user 0.001s sys 0.000s
W20260807 01:47:46.967514 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 87777.2us
W20260807 01:47:46.975942 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 99403.7us
I20260807 01:47:46.984354 27218 raft_consensus.cc:3060] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:46.934764 30954 raft_consensus.cc:2749] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:46.990509 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:46.990073 27101 raft_consensus.cc:2368] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3.
I20260807 01:47:46.988384 30865 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:46.993414 27227 raft_consensus.cc:3060] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:46.939036 30938 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:46.996724 27229 raft_consensus.cc:2368] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6.
I20260807 01:47:47.004246 30903 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:46.980538 26952 rpcz_store.cc:276] 0807 01:47:45.127352 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:45.127517 (+ 165us) service_pool.cc:224] Handling call
0807 01:47:46.953682 (+1826165us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 01:47:45.399363 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 01:47:45.400009 (+ 646us) write_op.cc:432] Acquiring schema lock in shared mode
0807 01:47:45.400039 (+ 30us) write_op.cc:435] Acquired schema lock
0807 01:47:45.400062 (+ 23us) tablet.cc:662] Decoding operations
0807 01:47:45.400512 (+ 450us) write_op.cc:620] Acquiring the partition lock for write op
0807 01:47:45.400572 (+ 60us) write_op.cc:641] Partition lock acquired for write op
0807 01:47:45.400602 (+ 30us) tablet.cc:685] Acquiring locks for 2 operations
0807 01:47:45.401027 (+ 425us) tablet.cc:701] Row locks acquired
0807 01:47:45.401047 (+ 20us) write_op.cc:260] PREPARE: finished
0807 01:47:45.407803 (+ 6756us) write_op.cc:270] Start()
0807 01:47:45.413256 (+ 5453us) write_op.cc:276] Timestamp: P: 1786067265401152 usec, L: 0
0807 01:47:45.413698 (+ 442us) op_driver.cc:348] REPLICATION: starting
0807 01:47:45.853819 (+440121us) log.cc:844] Serialized 901 byte log entry
0807 01:47:45.862581 (+ 8762us) op_driver.cc:464] REPLICATION: finished
0807 01:47:45.872022 (+ 9441us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b54002ffdd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541a208 00007f7d58e73f61 00007f7d58e73738 00007f7d58e783d1 00007f7d58e78389 00007f7d58e78319 00007f7d58e782e1 00007f7d58e7750d 00000000003aa554 00000000003aa399
0807 01:47:46.067491 (+195469us) write_op.cc:301] APPLY: starting
0807 01:47:46.073707 (+ 6216us) tablet.cc:1367] starting BulkCheckPresence
0807 01:47:46.107743 (+ 34036us) tablet.cc:1370] finished BulkCheckPresence
0807 01:47:46.109823 (+ 2080us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 01:47:46.159567 (+ 49744us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 01:47:46.170024 (+ 10457us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0807 01:47:46.174931 (+ 4907us) write_op.cc:312] APPLY: finished
0807 01:47:46.190032 (+ 15101us) log.cc:844] Serialized 29 byte log entry
0807 01:47:46.211683 (+ 21651us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:47:46.211775 (+ 92us) write_op.cc:454] Released schema lock
0807 01:47:46.211827 (+ 52us) write_op.cc:341] FINISH: Updating metrics
0807 01:47:46.212269 (+ 442us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:47:46.212299 (+ 30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":237243,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":181640,"apply.run_cpu_time_us":3061,"apply.run_wall_time_us":152067,"num_ops":2,"prepare.queue_time_us":197206,"prepare.run_cpu_time_us":10347,"prepare.run_wall_time_us":472846,"replication_time_us":448697,"spinlock_wait_cycles":1001157376,"thread_start_us":83837,"threads_started":1}]]}
I20260807 01:47:47.014768 30888 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:47.027669 30980 consensus_queue.cc:1048] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.009s
I20260807 01:47:47.028582 27099 raft_consensus.cc:1275] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260807 01:47:47.008335 30779 consensus_queue.cc:1048] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260807 01:47:47.029925 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:47.030645 27109 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:47.030665 27103 raft_consensus.cc:1275] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 01:47:47.008455 30854 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:47.035142 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:47.045991 27336 raft_consensus.cc:2393] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 4: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:47.035472 30888 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:47.037650 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:47.037868 30942 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:47.038582 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:47.046870 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:47.049033 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:47.049268 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:47.051715 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:47.053010 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:47.054340 30900 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:47.055615 27105 raft_consensus.cc:1275] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260807 01:47:47.056450 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:47.056627 30892 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:47.064950 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:47.090636 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:47.092065 27337 raft_consensus.cc:2368] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6.
I20260807 01:47:47.091792 27064 leader_election.cc:400] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6.
I20260807 01:47:47.092784 27064 leader_election.cc:403] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260807 01:47:47.068029 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:46.994876 30787 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:47.069007 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:47.055644 30949 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:47.071002 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:47.069734 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:47.105643 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:47.110879 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:47.111685 27102 raft_consensus.cc:2393] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 4: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:47:47.071017 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:47.081024 27333 raft_consensus.cc:3060] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:47.081264 27331 raft_consensus.cc:3060] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:47.112239 27325 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Advancing to term 9
I20260807 01:47:47.081017 27230 raft_consensus.cc:2468] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:47.087762 27109 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:47:47.093950 27108 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:47.091777 27099 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:47:47.084445 27101 raft_consensus.cc:2393] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 3: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:47:47.068841 30865 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:47.099395 27218 raft_consensus.cc:2468] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:47:47.107810 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:47.111006 27330 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 3
I20260807 01:47:47.111747 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:47.111629 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:47.114789 27327 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 5
I20260807 01:47:47.095012 28660 leader_election.cc:304] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:47.116514 27110 raft_consensus.cc:2368] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 3. Current term is 4.
I20260807 01:47:47.116334 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:47.116900 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:47.117070 27328 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 6
I20260807 01:47:47.122635 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:47.135341 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:47.062009 30856 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:47.138916 27172 leader_election.cc:400] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 3. Current term is 4.
I20260807 01:47:47.132565 27064 leader_election.cc:304] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:47.139191 27214 raft_consensus.cc:2368] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 5. Current term is 6.
I20260807 01:47:47.137420 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:47.143529 27230 raft_consensus.cc:2393] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 3: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:47.139755 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:47.145300 27338 raft_consensus.cc:2393] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 5: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:47.139987 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:47.142432 27172 leader_election.cc:403] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260807 01:47:47.150575 27101 raft_consensus.cc:2368] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3.
I20260807 01:47:47.151108 27331 raft_consensus.cc:2468] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:47.164681 28660 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:47.171584 27227 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:47:47.136656 30813 raft_consensus.cc:2749] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:47.180962 27333 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:47:47.181135 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:47.182024 27331 raft_consensus.cc:2368] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 3. Current term is 4.
I20260807 01:47:47.183005 27222 raft_consensus.cc:1240] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260807 01:47:47.138101 30681 raft_consensus.cc:2749] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6.
I20260807 01:47:47.186654 27338 raft_consensus.cc:1275] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260807 01:47:47.189729 28655 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:47.199033 27105 raft_consensus.cc:2393] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 6: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:47.180302 30783 raft_consensus.cc:2749] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 3. Current term is 4.
I20260807 01:47:47.206583 30972 raft_consensus.cc:2749] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 01:47:47.206935 30854 raft_consensus.cc:493] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:47.207485 30854 raft_consensus.cc:515] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:47.210510 27103 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:47:47.219641 27099 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 9.
I20260807 01:47:47.240985 30975 raft_consensus.cc:2749] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:47.241838 30927 raft_consensus.cc:2804] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:47:47.242456 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:47.243139 27336 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:47.244544 27287 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:47.247016 27287 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
W20260807 01:47:47.248852 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 256509us
I20260807 01:47:47.260088 27325 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 9.
I20260807 01:47:47.268831 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:47.248162 30927 raft_consensus.cc:697] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:47.257906 30888 raft_consensus.cc:2804] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:47:47.271729 27287 leader_election.cc:400] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3.
I20260807 01:47:47.271106 27325 raft_consensus.cc:2393] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 6: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:47.256479 30865 raft_consensus.cc:2804] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:47:47.279404 27287 leader_election.cc:403] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260807 01:47:47.282799 30970 consensus_queue.cc:1048] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:47.264758 30783 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:47.294649 30783 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:47.296993 30783 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:47.297104 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:47.297950 27212 raft_consensus.cc:2393] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 6: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:47.299202 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:47.299373 27212 raft_consensus.cc:1275] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260807 01:47:47.300021 27325 raft_consensus.cc:2368] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3.
I20260807 01:47:47.302289 30865 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:47.302713 30865 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:47.303862 30854 leader_election.cc:290] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:47.304785 30787 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:47.313504 30859 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:47.324792 27336 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:47.332115 27327 raft_consensus.cc:2468] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:47:47.336421 30888 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:47.338397 27108 raft_consensus.cc:2468] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:47:47.353186 27330 raft_consensus.cc:2468] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:47:47.365756 30896 raft_consensus.cc:2749] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3.
I20260807 01:47:47.366642 30888 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
W20260807 01:47:47.434938 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:47.433969 27328 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:47:47.424680 30901 raft_consensus.cc:3060] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
W20260807 01:47:47.515987 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 229412us
I20260807 01:47:47.287889 30927 consensus_queue.cc:237] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:47.539480 27214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57654 (request call id 1789) took 1060 ms. Trace:
I20260807 01:47:47.540557 27214 rpcz_store.cc:276] 0807 01:47:46.479106 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:46.484295 (+ 5189us) service_pool.cc:224] Handling call
0807 01:47:47.536650 (+1052355us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b54000f23d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:47:47.539371 (+ 2721us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1487758080}
I20260807 01:47:47.529464 30865 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:47.601636 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 409134us
I20260807 01:47:47.580585 30776 consensus_queue.cc:1048] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
W20260807 01:47:47.614367 30992 log.cc:927] Time spent T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.075s user 0.001s sys 0.000s
I20260807 01:47:47.615688 27214 raft_consensus.cc:1275] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 01:47:47.619930 26950 catalog_manager.cc:5697] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 4 to 5, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 5 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:47.632061 27222 raft_consensus.cc:1275] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 01:47:47.596808 31003 consensus_queue.cc:1048] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260807 01:47:47.635797 30888 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:47.638500 31002 log.cc:927] Time spent T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.061s user 0.001s sys 0.000s
I20260807 01:47:47.642068 30888 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:47.647543 28660 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:47.677660 30856 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:47.672501 30928 raft_consensus.cc:2749] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:47.687476 30865 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:47.690145 30736 consensus_queue.cc:1048] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:47:47.702521 30938 raft_consensus.cc:493] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
W20260807 01:47:47.706715 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 151863us
I20260807 01:47:47.709699 30940 consensus_queue.cc:1048] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:47:47.711158 27328 raft_consensus.cc:1275] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260807 01:47:47.718676 27328 pending_rounds.cc:85] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade: Aborting all ops after (but not including) 4
I20260807 01:47:47.718962 27328 pending_rounds.cc:107] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260807 01:47:47.719199 27328 raft_consensus.cc:2889] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 01:47:47.717800 27287 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:47.728359 27063 leader_election.cc:304] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:47.730309 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:47.731123 27226 raft_consensus.cc:2368] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 3. Current term is 4.
I20260807 01:47:47.731132 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:47.732590 27110 raft_consensus.cc:2368] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 3. Current term is 4.
I20260807 01:47:47.733065 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:47.728639 27170 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:47.733796 27108 raft_consensus.cc:2393] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 4: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:47:47.735358 27063 leader_election.cc:400] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Vote denied by peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429) with higher term. Message: Invalid argument: T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6.
I20260807 01:47:47.736294 27063 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:47.738819 27170 leader_election.cc:304] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:47.740967 27170 leader_election.cc:400] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Vote denied by peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429) with higher term. Message: Invalid argument: T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 3. Current term is 4.
I20260807 01:47:47.742914 27172 leader_election.cc:400] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3.
I20260807 01:47:47.743274 27172 leader_election.cc:403] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260807 01:47:47.743777 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:47.744238 27063 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:47.744536 27170 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:47.746918 27222 raft_consensus.cc:2393] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 4: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:47:47.742239 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:47.748476 27218 raft_consensus.cc:2393] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 4: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:47.747105 30854 consensus_queue.cc:1048] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260807 01:47:47.741932 30783 raft_consensus.cc:2749] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:47.740370 30896 raft_consensus.cc:2804] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:47:47.752203 30896 raft_consensus.cc:697] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:47.753131 30896 consensus_queue.cc:237] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:47.758059 27064 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:47:47.742441 30941 raft_consensus.cc:2749] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:47.740732 30801 raft_consensus.cc:2749] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:47.740577 30856 raft_consensus.cc:2749] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:47.781878 27104 raft_consensus.cc:1275] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 01:47:47.773746 30937 raft_consensus.cc:2804] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election won for term 3
W20260807 01:47:47.789050 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 46277us
I20260807 01:47:47.790306 27172 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:47:47.742489 30938 raft_consensus.cc:515] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:47.789474 30854 raft_consensus.cc:2749] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3.
I20260807 01:47:47.797942 30783 raft_consensus.cc:2804] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader election won for term 9
I20260807 01:47:47.808845 30937 raft_consensus.cc:697] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:47.819609 30937 consensus_queue.cc:237] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:47.802987 30856 raft_consensus.cc:2804] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election won for term 5
W20260807 01:47:47.831579 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 87434.7us
W20260807 01:47:47.832144 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 74157.8us
I20260807 01:47:47.832840 27063 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
W20260807 01:47:47.833302 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:47.835168 27170 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:47.835345 27172 leader_election.cc:304] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:47.837145 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:47.848035 27101 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:47.808825 30783 raft_consensus.cc:697] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 9 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:47.865542 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:47.858492 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:47.912981 27101 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:47:47.864377 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:47.837785 30778 consensus_queue.cc:1059] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260807 01:47:47.836723 30859 raft_consensus.cc:2804] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:47:47.857928 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:47.917122 30778 raft_consensus.cc:3055] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 01:47:47.917553 30778 raft_consensus.cc:740] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:47.918501 30778 consensus_queue.cc:260] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:47.854087 30970 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:47.859226 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:47.920171 30970 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:47.850908 30854 raft_consensus.cc:2749] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 01:47:47.869555 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:47.829397 30856 raft_consensus.cc:697] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:47.925858 30999 consensus_queue.cc:1048] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:47:47.837986 30941 raft_consensus.cc:2804] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:47.850103 27170 leader_election.cc:400] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 2 election: Vote denied by peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429) with higher term. Message: Invalid argument: T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3.
W20260807 01:47:47.928897 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 78961.2us
I20260807 01:47:47.911943 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:47.869846 27287 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:47.824682 30736 consensus_queue.cc:1048] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
W20260807 01:47:47.930747 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 60979.5us
I20260807 01:47:47.919872 30778 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:47.911217 30787 raft_consensus.cc:2804] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:47:47.934254 30938 leader_election.cc:290] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:47.947105 27287 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:47.962142 31007 raft_consensus.cc:493] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
W20260807 01:47:47.972420 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:47.975212 27287 leader_election.cc:400] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3.
I20260807 01:47:47.994756 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:47.928611 30859 raft_consensus.cc:697] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
W20260807 01:47:48.007524 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 77642.7us
I20260807 01:47:47.998588 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:47.938871 30979 raft_consensus.cc:2749] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:47:48.015511 27287 leader_election.cc:400] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 8. Current term is 9.
I20260807 01:47:48.018193 27287 leader_election.cc:403] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260807 01:47:47.970791 30944 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:47.962927 31007 raft_consensus.cc:515] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:48.024030 27287 leader_election.cc:400] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5.
W20260807 01:47:48.028487 31020 log.cc:927] Time spent T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.059s user 0.000s sys 0.001s
I20260807 01:47:48.027704 27287 leader_election.cc:403] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260807 01:47:47.970585 30787 raft_consensus.cc:697] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:47.997840 30865 raft_consensus.cc:2804] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:48.026652 27067 leader_election.cc:304] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:48.037436 27170 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:48.017928 30942 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:48.019430 30779 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:48.038874 30711 raft_consensus.cc:2749] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 8. Current term is 9.
I20260807 01:47:48.036684 30888 raft_consensus.cc:2749] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5.
I20260807 01:47:48.088196 30970 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:48.040593 30856 consensus_queue.cc:237] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:48.088881 30787 consensus_queue.cc:237] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:48.032590 30944 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:48.089747 30859 consensus_queue.cc:237] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:48.080102 30942 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:48.033914 30783 consensus_queue.cc:237] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:48.096443 27099 raft_consensus.cc:1275] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:47:48.055454 30801 raft_consensus.cc:2749] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:48.056684 30865 raft_consensus.cc:697] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:48.098913 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:47:48.100500 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 63157.3us
I20260807 01:47:48.110786 27170 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
W20260807 01:47:48.102198 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 64091.1us
I20260807 01:47:48.102699 30708 raft_consensus.cc:2804] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:48.117946 27287 leader_election.cc:400] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 5. Current term is 6.
I20260807 01:47:48.118319 27287 leader_election.cc:403] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260807 01:47:48.100054 30854 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:48.127863 27287 leader_election.cc:400] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 3. Current term is 4.
I20260807 01:47:48.128328 27287 leader_election.cc:403] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260807 01:47:48.079470 30779 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:48.158191 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:48.159488 27325 raft_consensus.cc:1275] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 01:47:48.159718 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:48.107040 30865 consensus_queue.cc:237] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:48.161026 27325 raft_consensus.cc:1240] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260807 01:47:48.162597 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:48.162793 27330 raft_consensus.cc:1275] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 01:47:48.163362 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:48.121003 30854 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:48.169919 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:48.119055 30970 consensus_queue.cc:1048] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260807 01:47:48.150776 30711 raft_consensus.cc:2749] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 5. Current term is 6.
I20260807 01:47:48.165328 30888 raft_consensus.cc:2749] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 3. Current term is 4.
I20260807 01:47:48.193701 31026 raft_consensus.cc:2804] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:47:48.223889 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:48.260371 27287 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:48.275231 30708 raft_consensus.cc:697] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:48.276440 30708 consensus_queue.cc:237] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:48.224538 30995 log.cc:927] Time spent T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.057s user 0.001s sys 0.000s
I20260807 01:47:48.302409 27326 raft_consensus.cc:2393] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 6: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:47:48.303648 27287 leader_election.cc:400] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 3. Current term is 4.
W20260807 01:47:48.232400 30985 log.cc:927] Time spent T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.053s user 0.001s sys 0.000s
I20260807 01:47:48.308183 27287 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:47:48.231547 30942 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:48.226703 31026 raft_consensus.cc:697] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:48.315414 31003 raft_consensus.cc:2804] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:47:48.316021 31003 raft_consensus.cc:697] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:48.336757 31003 consensus_queue.cc:237] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:48.342767 31030 log.cc:927] Time spent T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.138s user 0.001s sys 0.000s
I20260807 01:47:48.305603 30927 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:48.347199 30927 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:48.349447 30927 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:47:48.353394 31016 log.cc:927] Time spent T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.231s user 0.002s sys 0.002s
I20260807 01:47:48.353642 31007 leader_election.cc:290] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:48.354887 30779 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:48.305945 30846 raft_consensus.cc:2749] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 01:47:48.359895 30854 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:48.419792 28713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48362 (request call id 690) took 1296 ms. Trace:
I20260807 01:47:48.420295 28713 rpcz_store.cc:276] 0807 01:47:47.123819 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:47.134884 (+ 11065us) service_pool.cc:224] Handling call
0807 01:47:47.137192 (+ 2308us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:47:47.140194 (+ 3002us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:47.140242 (+ 48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:47:47.808585 (+668343us) spinlock_profiling.cc:243] Waited 6.58 ms on lock 0x7b2c0001bfb8. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d53d19c71 00007f7d53d295d2 00007f7d55459de1 00007f7d5541e81a 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22
0807 01:47:47.810468 (+ 1883us) log.cc:844] Serialized 26 byte log entry
0807 01:47:47.994433 (+183965us) raft_consensus.cc:1645] Marking committed up to 8
0807 01:47:48.007503 (+ 13070us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:48.016466 (+ 8963us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:47:48.352760 (+336294us) spinlock_profiling.cc:243] Waited 6.54 ms on lock 0x7b440002250c. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d5223e855 00007f7d5223f272 00007f7d54fc93bb 00007f7d54fc4c9f 00007f7d54fb8a5e 00007f7d54fb7da6 00007f7d54fb79c8 00007f7d54fc1411 00007f7d54fc13c9 00007f7d54fc1359
0807 01:47:48.410794 (+ 58034us) raft_consensus.cc:1685] finished
0807 01:47:48.419466 (+ 8672us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:48.419688 (+ 222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2553,"raft.queue_time_us":3039,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"spinlock_wait_cycles":30185984,"thread_start_us":132537,"threads_started":1,"wal-append.queue_time_us":269118}
I20260807 01:47:48.422310 31026 consensus_queue.cc:237] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:48.420995 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:48.428874 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:47:48.433640 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 125533us
I20260807 01:47:48.439242 27287 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
W20260807 01:47:48.440616 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 407570us
I20260807 01:47:48.451828 27287 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:48.447225 30779 raft_consensus.cc:2749] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260807 01:47:48.470937 30896 raft_consensus.cc:2749] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:47:48.563613 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:48.550925 30779 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:48.566075 30779 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:48.573508 30779 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:48.679986 28709 raft_consensus.cc:1275] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260807 01:47:49.087474 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 103294us
I20260807 01:47:49.089051 30779 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:49.145344 31003 raft_consensus.cc:493] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:49.102674 30901 consensus_queue.cc:1048] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.102s
I20260807 01:47:49.208078 30776 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:49.223696 30816 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:49.224442 30816 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:49.227950 30859 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s user 0.010s sys 0.027s
W20260807 01:47:49.198861 31026 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.628s user 0.006s sys 0.035s
W20260807 01:47:49.232342 30859 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s user 0.010s sys 0.027s
I20260807 01:47:49.210035 31003 raft_consensus.cc:515] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:49.199527 30779 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:49.233866 31026 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.663s user 0.006s sys 0.035s
I20260807 01:47:49.227584 30776 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
W20260807 01:47:49.241495 31007 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.740s user 0.015s sys 0.013s
I20260807 01:47:49.258193 30901 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:49.257869 30813 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
W20260807 01:47:49.262987 31007 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.761s user 0.015s sys 0.014s
I20260807 01:47:49.280160 31071 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:49.285135 31071 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:49.289175 31071 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:49.289880 31071 raft_consensus.cc:493] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:49.292374 31071 raft_consensus.cc:515] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:49.319653 31071 leader_election.cc:290] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:49.341183 30776 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:49.342743 31003 leader_election.cc:290] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:49.343627 31072 raft_consensus.cc:493] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:49.344370 31072 raft_consensus.cc:515] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:49.268882 30901 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:49.332712 31071 raft_consensus.cc:493] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:49.270329 30813 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:49.357009 31073 raft_consensus.cc:493] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:49.362370 30779 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:49.363467 31072 leader_election.cc:290] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:49.364696 27328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33976 (request call id 2457) took 1209 ms. Trace:
I20260807 01:47:49.360752 31071 raft_consensus.cc:515] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:49.365178 27328 rpcz_store.cc:276] 0807 01:47:48.155464 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:48.155581 (+ 117us) service_pool.cc:224] Handling call
0807 01:47:48.155926 (+ 345us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:48.194655 (+ 38729us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:48.194684 (+ 29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:48.194742 (+ 58us) raft_consensus.cc:1645] Marking committed up to 7
0807 01:47:48.194784 (+ 42us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:49.364254 (+1169470us) spinlock_profiling.cc:243] Waited 38.5 ms on lock 0x7b5400407c54. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:47:49.364317 (+ 63us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:49.364503 (+ 186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88514944}
I20260807 01:47:49.370723 30813 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:49.373076 31073 raft_consensus.cc:515] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:49.375712 30901 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:49.379953 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:49.383631 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:49.385037 28711 raft_consensus.cc:2468] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:47:49.381736 28715 raft_consensus.cc:2468] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:47:49.398224 27289 leader_election.cc:304] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:49.402040 31074 raft_consensus.cc:2804] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:49.402837 31074 raft_consensus.cc:493] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:49.403419 31074 raft_consensus.cc:3060] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:49.407300 31071 leader_election.cc:290] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:49.410944 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:49.397969 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:49.410630 31073 leader_election.cc:290] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:49.416679 31074 raft_consensus.cc:515] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:49.418742 28702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48362 (request call id 694) took 1382 ms. Trace:
I20260807 01:47:49.419927 31074 leader_election.cc:290] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:49.419423 28702 rpcz_store.cc:276] 0807 01:47:48.035800 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:48.044124 (+ 8324us) service_pool.cc:224] Handling call
0807 01:47:49.412936 (+1368812us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b5800055598. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e084c 00007f7d522e040b 00007f7d5b24c219 00007f7d5a39f640 00007f7d5a467ebb 00007f7d5a467f41 00007f7d5a429dd0 00007f7d5a414739 00007f7d5a4141e2 00007f7d53959d34
0807 01:47:49.413432 (+ 496us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:49.413841 (+ 409us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:49.413875 (+ 34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:49.413942 (+ 67us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:47:49.414001 (+ 59us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:49.414080 (+ 79us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:49.418634 (+ 4554us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5053696}
I20260807 01:47:49.428963 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:49.432091 31073 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:49.436398 31073 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:49.440054 31073 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:47:49.444623 27326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59490 (request call id 2471) took 1286 ms (client timeout 1711 ms). Trace:
W20260807 01:47:49.445293 27326 rpcz_store.cc:269] 0807 01:47:48.158203 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:48.158331 (+ 128us) service_pool.cc:224] Handling call
0807 01:47:49.444081 (+1285750us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54000221d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:47:49.444422 (+ 341us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":250625408}
I20260807 01:47:49.432834 28702 raft_consensus.cc:3060] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:49.453936 26950 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46138 (request call id 397) took 3106 ms. Trace:
I20260807 01:47:49.455835 26950 rpcz_store.cc:276] 0807 01:47:46.348001 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:46.351871 (+ 3870us) service_pool.cc:224] Handling call
0807 01:47:49.453773 (+3101902us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 01:47:47.799893 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 01:47:47.801303 (+ 1410us) write_op.cc:432] Acquiring schema lock in shared mode
0807 01:47:47.801345 (+ 42us) write_op.cc:435] Acquired schema lock
0807 01:47:47.801363 (+ 18us) tablet.cc:662] Decoding operations
0807 01:47:47.801698 (+ 335us) write_op.cc:620] Acquiring the partition lock for write op
0807 01:47:47.801777 (+ 79us) write_op.cc:641] Partition lock acquired for write op
0807 01:47:47.801808 (+ 31us) tablet.cc:685] Acquiring locks for 1 operations
0807 01:47:47.801993 (+ 185us) tablet.cc:701] Row locks acquired
0807 01:47:47.802010 (+ 17us) write_op.cc:260] PREPARE: finished
0807 01:47:47.802198 (+ 188us) write_op.cc:270] Start()
0807 01:47:47.802327 (+ 129us) write_op.cc:276] Timestamp: P: 1786067267802116 usec, L: 0
0807 01:47:47.802347 (+ 20us) op_driver.cc:348] REPLICATION: starting
0807 01:47:47.803076 (+ 729us) log.cc:844] Serialized 532 byte log entry
0807 01:47:48.392160 (+589084us) op_driver.cc:464] REPLICATION: finished
0807 01:47:48.554423 (+162263us) write_op.cc:301] APPLY: starting
0807 01:47:48.554517 (+ 94us) tablet.cc:1367] starting BulkCheckPresence
0807 01:47:48.554646 (+ 129us) tablet.cc:1370] finished BulkCheckPresence
0807 01:47:48.554763 (+ 117us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 01:47:48.555008 (+ 245us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 01:47:48.555225 (+ 217us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0807 01:47:48.555272 (+ 47us) write_op.cc:312] APPLY: finished
0807 01:47:48.555474 (+ 202us) log.cc:844] Serialized 23 byte log entry
0807 01:47:48.555648 (+ 174us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:47:48.555728 (+ 80us) write_op.cc:454] Released schema lock
0807 01:47:48.555767 (+ 39us) write_op.cc:341] FINISH: Updating metrics
0807 01:47:48.556197 (+ 430us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:47:48.556224 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":309640,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":161973,"apply.run_cpu_time_us":1762,"apply.run_wall_time_us":1763,"num_ops":1,"prepare.queue_time_us":146062,"prepare.run_cpu_time_us":42809,"prepare.run_wall_time_us":207685,"raft.queue_time_us":229066,"raft.run_cpu_time_us":13816,"raft.run_wall_time_us":55747,"replication_time_us":589576,"thread_start_us":427596,"threads_started":4,"wal-append.queue_time_us":285108}]]}
I20260807 01:47:49.477075 28702 raft_consensus.cc:2468] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
W20260807 01:47:49.490654 30941 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.474s user 0.013s sys 0.022s
W20260807 01:47:49.491549 30941 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.495s user 0.013s sys 0.022s
I20260807 01:47:49.494192 30941 raft_consensus.cc:697] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:49.502105 30941 consensus_queue.cc:237] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:49.522356 30962 raft_consensus.cc:493] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:49.523109 30962 raft_consensus.cc:515] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:49.525571 30962 leader_election.cc:290] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:47:49.819499 31055 log.cc:927] Time spent T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 1.233s user 0.004s sys 0.011s
I20260807 01:47:49.987391 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:49.988451 27109 raft_consensus.cc:2468] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:47:49.989794 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:50.064888 27109 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:47:50.066264 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:50.108911 27109 raft_consensus.cc:2468] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 2.
I20260807 01:47:50.231820 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:50.343298 31087 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:50.344026 31087 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.353880 31089 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:50.355260 31089 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.359141 31089 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:47:50.377322 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 1.93361e+06us
W20260807 01:47:50.389134 27063 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s user 0.002s sys 0.005s
W20260807 01:47:50.389916 27063 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s user 0.002s sys 0.005s
I20260807 01:47:50.390678 31089 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:50.391390 31089 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.393464 31089 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:50.394431 31089 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:50.394551 31087 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:50.394994 31089 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:50.395798 31087 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:50.396893 31087 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:50.397490 31089 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:50.398074 31089 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:50.398530 31089 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:50.399288 31087 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:50.400048 31090 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:50.400727 31090 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.401088 31089 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:50.402320 31087 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:50.402889 31087 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:50.405007 31087 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:50.405547 31089 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:50.406234 31089 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.408968 31089 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:50.409178 31090 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:47:50.413918 27063 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 2.194s (SENT) [suppressed 2 similar messages]
I20260807 01:47:50.414392 31091 raft_consensus.cc:493] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:50.415134 31091 raft_consensus.cc:515] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.416564 31090 consensus_queue.cc:1048] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:47:50.420120 31091 leader_election.cc:290] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:50.426191 30970 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:50.426903 30970 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.428622 30970 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:50.431519 31091 raft_consensus.cc:493] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:50.432196 31091 raft_consensus.cc:515] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.434665 31091 leader_election.cc:290] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:50.435297 31091 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:50.434542 31101 raft_consensus.cc:493] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:50.435776 31091 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:50.436159 31101 raft_consensus.cc:515] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:50.438024 31091 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:50.439369 31101 leader_election.cc:290] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:50.440485 31101 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:50.440609 31089 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:50.441501 31101 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.441958 31003 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:50.442374 31089 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.442946 31003 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:50.445436 31087 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:50.446130 31089 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:50.446132 31003 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:50.447937 31087 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:50.448729 31090 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:50.450932 31089 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:50.447507 31101 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:50.449795 31003 raft_consensus.cc:493] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:50.458513 31089 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:50.460839 31003 raft_consensus.cc:515] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.456099 31103 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:50.469481 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:50.470616 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:50.471019 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:50.479427 28707 raft_consensus.cc:2468] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:50.472920 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:50.454401 31090 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.476161 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:50.479620 27109 raft_consensus.cc:1275] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 01:47:50.481626 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:50.481664 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:50.477161 28714 raft_consensus.cc:2468] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:50.471678 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:50.481271 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:50.467486 31087 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:50.491622 31089 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:50.492331 31090 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:50.492698 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:50.485215 31103 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.504040 31103 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:50.505714 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:50.506093 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:50.506767 27103 raft_consensus.cc:2468] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 7.
I20260807 01:47:50.508111 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:50.508765 27103 raft_consensus.cc:2468] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:47:50.508926 31003 leader_election.cc:290] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:50.510124 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:50.511814 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:50.512408 27103 raft_consensus.cc:2468] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:47:50.513351 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:50.513979 27103 raft_consensus.cc:2468] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:47:50.535426 28707 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:34614
I20260807 01:47:50.536826 28707 raft_consensus.cc:606] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:50.542745 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260807 01:47:50.543883 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:50.544549 28715 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:47:50.544811 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:47:50.546595 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:50.549649 27102 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 527) took 1027 ms. Trace:
I20260807 01:47:50.550119 27102 rpcz_store.cc:276] 0807 01:47:49.521925 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:50.548777 (+1026852us) service_pool.cc:224] Handling call
0807 01:47:50.549109 (+ 332us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:50.549319 (+ 210us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:50.549344 (+ 25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:50.549384 (+ 40us) raft_consensus.cc:1645] Marking committed up to 5
0807 01:47:50.549426 (+ 42us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:50.549477 (+ 51us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:50.549587 (+ 110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:50.552763 27100 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 525) took 1031 ms. Trace:
I20260807 01:47:50.553305 27100 rpcz_store.cc:276] 0807 01:47:49.521319 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:50.551926 (+1030607us) service_pool.cc:224] Handling call
0807 01:47:50.552236 (+ 310us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:50.552426 (+ 190us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:50.552451 (+ 25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:50.552490 (+ 39us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:47:50.552535 (+ 45us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:50.552584 (+ 49us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:50.552701 (+ 117us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 01:47:50.553498 27105 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41080 (request call id 2880) took 2696 ms (client timeout 1711 ms). Trace:
W20260807 01:47:50.554780 27105 rpcz_store.cc:269] 0807 01:47:47.857132 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:47.865228 (+ 8096us) service_pool.cc:224] Handling call
0807 01:47:50.553191 (+2687963us) spinlock_profiling.cc:243] Waited 2.64 s on lock 0x7b54003183d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:47:50.553323 (+ 132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6066136832}
I20260807 01:47:50.559504 28713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52322 (request call id 587) took 2134 ms. Trace:
I20260807 01:47:50.560786 27333 raft_consensus.cc:2393] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 4: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:50.560107 28713 rpcz_store.cc:276] 0807 01:47:48.424881 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:50.557922 (+2133041us) service_pool.cc:224] Handling call
0807 01:47:50.558746 (+ 824us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:50.559087 (+ 341us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:50.559143 (+ 56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:50.559205 (+ 62us) raft_consensus.cc:1645] Marking committed up to 5
0807 01:47:50.559253 (+ 48us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:50.559315 (+ 62us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:50.559439 (+ 124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:50.578485 27099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 524) took 1057 ms. Trace:
I20260807 01:47:50.579002 27099 rpcz_store.cc:276] 0807 01:47:49.520885 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:50.577281 (+1056396us) service_pool.cc:224] Handling call
0807 01:47:50.577854 (+ 573us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:50.578080 (+ 226us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:50.578119 (+ 39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:50.578188 (+ 69us) raft_consensus.cc:1645] Marking committed up to 7
0807 01:47:50.578234 (+ 46us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:50.578291 (+ 57us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:50.578417 (+ 126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:50.587607 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:50.587632 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:50.591297 30888 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:50.591917 27102 raft_consensus.cc:2468] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:47:50.588347 27100 raft_consensus.cc:2468] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:47:50.590891 30779 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:50.593315 30779 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:50.597033 30888 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:50.597707 27287 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:50.599565 27287 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:50.601264 30888 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:50.601562 31003 consensus_queue.cc:1048] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:47:50.602038 30888 raft_consensus.cc:2749] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:47:50.603046 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:47:50.603438 27101 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.172s user 0.030s sys 0.023s
W20260807 01:47:50.605314 27101 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.174s user 0.030s sys 0.024s
I20260807 01:47:50.605433 27105 raft_consensus.cc:2468] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:47:50.604301 30888 raft_consensus.cc:2804] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:47:50.600068 27338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36604 (request call id 574) took 2273 ms. Trace:
I20260807 01:47:50.607105 27338 rpcz_store.cc:276] 0807 01:47:48.326695 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:48.327024 (+ 329us) service_pool.cc:224] Handling call
0807 01:47:48.327521 (+ 497us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:47:48.336800 (+ 9279us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:48.336837 (+ 37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:47:48.337415 (+ 578us) log.cc:844] Serialized 26 byte log entry
0807 01:47:50.596060 (+2258645us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b1800383920. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d5b233f31 00007f7d5b2406d1 00007f7d54fc56ac 00007f7d522fae3b 00007f7d5b1a2fc9 00007f7d52316fdb 00007f7d523152db 00007f7d52317542 00007f7d54fb73fb 00007f7d54fbcf13 00007f7d54fbd2c1
0807 01:47:50.596305 (+ 245us) raft_consensus.cc:1645] Marking committed up to 4
0807 01:47:50.596345 (+ 40us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:50.596435 (+ 90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:47:50.596475 (+ 40us) raft_consensus.cc:1685] finished
0807 01:47:50.596487 (+ 12us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:50.599959 (+ 3472us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5338,"spinlock_wait_cycles":99100800,"thread_start_us":46884,"threads_started":1,"wal-append.queue_time_us":88826,"wal-append.run_cpu_time_us":2237,"wal-append.run_wall_time_us":1028182}
I20260807 01:47:50.604640 30779 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:50.600495 30970 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:47:50.613443 27333 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33976 (request call id 2456) took 2458 ms (client timeout 2194 ms). Trace:
I20260807 01:47:50.612883 30970 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:50.613889 27333 rpcz_store.cc:269] 0807 01:47:48.155162 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:48.155274 (+ 112us) service_pool.cc:224] Handling call
0807 01:47:50.613106 (+2457832us) spinlock_profiling.cc:243] Waited 2.4 s on lock 0x7b540001dbd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:47:50.613291 (+ 185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5513054976}
I20260807 01:47:50.611781 30888 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:50.591013 30875 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:50.606418 31102 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:50.616285 30875 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.616992 31102 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.616211 27101 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1629) took 2443 ms. Trace:
I20260807 01:47:50.619491 27101 rpcz_store.cc:276] 0807 01:47:48.168371 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:48.168506 (+ 135us) service_pool.cc:224] Handling call
0807 01:47:48.168826 (+ 320us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:47:48.429411 (+260585us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:48.429442 (+ 31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:47:48.430956 (+ 1514us) log.cc:844] Serialized 26 byte log entry
0807 01:47:49.818747 (+1387791us) spinlock_profiling.cc:243] Waited 36.3 ms on lock 0x7b440002250c. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d5223e855 00007f7d5223f272 00007f7d54fc93bb 00007f7d54fc4c9f 00007f7d54fb8a5e 00007f7d54fb7da6 00007f7d54fb79c8 00007f7d54fc1411 00007f7d54fc13c9 00007f7d54fc1359
0807 01:47:50.605728 (+786981us) raft_consensus.cc:1645] Marking committed up to 8
0807 01:47:50.605758 (+ 30us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:50.605822 (+ 64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:47:50.605864 (+ 42us) raft_consensus.cc:1685] finished
0807 01:47:50.605874 (+ 10us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:50.611636 (+ 5762us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14675,"raft.queue_time_us":57365,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":9513,"spinlock_wait_cycles":83597696,"thread_start_us":2410959,"threads_started":2,"wal-append.queue_time_us":149736}
I20260807 01:47:50.615286 30888 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:50.626989 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:50.626565 30875 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:50.629002 31112 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:50.630312 31112 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:50.643961 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 44450.6us
I20260807 01:47:50.646796 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:50.665810 31112 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:50.669659 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:50.671442 27287 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:50.673596 27287 leader_election.cc:304] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:50.675372 27287 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:50.675181 30779 raft_consensus.cc:2804] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:47:50.681552 30779 raft_consensus.cc:493] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:50.682278 30779 raft_consensus.cc:3060] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:50.686477 31007 raft_consensus.cc:2804] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:47:50.687196 31007 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:50.687664 31007 raft_consensus.cc:3060] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:50.674381 30962 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:50.690804 31068 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:50.691725 31068 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.694954 31116 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:50.696007 31116 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.674875 30875 raft_consensus.cc:2804] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:47:50.698671 30875 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:50.699280 30875 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:50.690694 30962 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:50.706571 31111 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
W20260807 01:47:50.708297 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 88332.2us
I20260807 01:47:50.709481 31068 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:47:50.710407 31113 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:50.716939 30962 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:50.729226 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:47:50.730377 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 53483.3us
I20260807 01:47:50.732933 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:50.711404 31113 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.734261 31118 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:50.740025 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:50.741643 30846 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:50.742419 30846 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:50.745581 30846 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:50.748896 31117 raft_consensus.cc:493] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:50.748436 31007 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:50.751163 30816 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:50.756148 31116 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:50.758667 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:50.713107 31111 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.764828 30944 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:50.762425 27067 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:50.753120 26948 catalog_manager.cc:5697] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 2 to 3, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4). New cstate: current_term: 3 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:50.776980 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:50.775116 26948 catalog_manager.cc:5697] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:50.777737 27338 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
I20260807 01:47:50.777472 27067 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:50.782485 28655 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:50.787727 27067 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:50.775892 30846 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:50.779301 30944 raft_consensus.cc:2804] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260807 01:47:50.793700 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 62338.5us
I20260807 01:47:50.794435 27104 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 526) took 1272 ms. Trace:
I20260807 01:47:50.783710 30778 raft_consensus.cc:2804] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:47:50.749636 31117 raft_consensus.cc:515] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.794762 27104 rpcz_store.cc:276] 0807 01:47:49.521637 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:50.785902 (+1264265us) service_pool.cc:224] Handling call
0807 01:47:50.793966 (+ 8064us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:50.794114 (+ 148us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:50.794134 (+ 20us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:50.794222 (+ 88us) raft_consensus.cc:1645] Marking committed up to 5
0807 01:47:50.794263 (+ 41us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:50.794302 (+ 39us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:50.794386 (+ 84us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:50.735322 31118 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.790513 31090 raft_consensus.cc:2804] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:47:50.799918 31090 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:50.800490 31090 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:50.789179 31116 raft_consensus.cc:2804] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:47:50.797494 30896 raft_consensus.cc:493] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:50.807027 31113 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:50.792510 30944 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:50.808105 30944 raft_consensus.cc:3060] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:50.808763 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:50.798946 31115 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:50.793998 30846 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:50.796068 30886 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:50.810289 31115 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.826947 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:50.873241 30888 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:50.806926 30896 raft_consensus.cc:515] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:50.937445 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 142989us
I20260807 01:47:50.836145 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:51.135699 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:51.149462 27104 raft_consensus.cc:2468] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 2.
I20260807 01:47:51.145843 27287 leader_election.cc:304] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:50.796057 30778 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:51.113330 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:51.167667 27326 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:47:50.912247 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:47:50.947466 27265 debug-util.cc:398] Leaking SignalData structure 0x7b080027baa0 after lost signal to thread 24600
I20260807 01:47:50.913123 28709 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Advancing to term 6
I20260807 01:47:50.888013 30865 raft_consensus.cc:493] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:51.143322 30711 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:50.838043 30846 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:47:51.176560 27265 debug-util.cc:398] Leaking SignalData structure 0x7b080015ef20 after lost signal to thread 27011
I20260807 01:47:50.837340 31007 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:51.192421 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:51.192799 30865 raft_consensus.cc:515] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.880602 31117 leader_election.cc:290] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:50.811443 31116 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:51.186764 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:50.814625 31113 raft_consensus.cc:493] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:50.948072 31118 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:50.826992 31119 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:51.169459 30778 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:51.041520 30875 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:50.877272 30779 raft_consensus.cc:515] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:50.814636 30886 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:51.208343 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 62607.1us
I20260807 01:47:51.209162 27287 leader_election.cc:304] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:50.856211 31111 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:47:51.207123 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 326053us
I20260807 01:47:51.201573 31116 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:51.201159 31007 raft_consensus.cc:2804] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:47:51.211162 27287 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:51.188235 30711 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:51.203181 31122 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:51.220575 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:51.222666 27337 raft_consensus.cc:1240] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 3. Current term is 4. Ops: []
I20260807 01:47:51.223091 27330 raft_consensus.cc:2468] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:51.224804 31115 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:51.224987 30888 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:51.227926 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:51.228819 28709 raft_consensus.cc:1275] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [term 6 LEARNER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260807 01:47:51.230657 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:51.233660 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:51.232128 31122 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:51.236016 31122 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:51.237007 30711 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:51.213824 30962 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:51.237404 28655 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:51.237902 30962 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:51.239044 30886 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:51.238871 31115 raft_consensus.cc:2804] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:47:51.239993 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:51.207231 31119 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:51.213860 31124 raft_consensus.cc:2804] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:47:51.240646 27104 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 8.
I20260807 01:47:51.239677 31115 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:51.243014 30962 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:51.242877 31090 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:51.244931 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:51.247001 30865 leader_election.cc:290] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:51.258539 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:51.251973 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:51.251612 27289 leader_election.cc:304] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:51.252445 31102 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:51.253108 30896 leader_election.cc:290] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:51.253659 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:51.213770 31113 raft_consensus.cc:515] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:51.254405 30875 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:51.255393 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:51.223155 31003 raft_consensus.cc:2804] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:51.258235 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:51.271019 31003 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:51.271374 27328 raft_consensus.cc:2393] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 4: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:51.256117 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:51.272150 27325 raft_consensus.cc:2368] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 3. Current term is 4.
I20260807 01:47:51.221709 31007 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:51.272929 31007 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:51.257079 31115 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:51.274135 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:51.274893 28710 raft_consensus.cc:2468] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 1.
I20260807 01:47:51.261947 30865 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:51.276434 30865 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:51.277619 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:51.271484 31003 raft_consensus.cc:3060] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:51.278719 30865 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:51.262259 30779 leader_election.cc:290] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:51.264611 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:51.280764 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:51.263737 27326 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:51.267297 27333 raft_consensus.cc:2468] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
I20260807 01:47:51.268784 31119 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:51.269958 31113 leader_election.cc:290] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:51.270241 31090 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:51.250546 31124 raft_consensus.cc:493] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:51.263278 30711 raft_consensus.cc:2804] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:47:51.285796 31124 raft_consensus.cc:3060] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:51.286252 30711 raft_consensus.cc:697] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:51.286935 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:51.287794 27336 raft_consensus.cc:2368] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 3. Current term is 4.
W20260807 01:47:51.288300 27336 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33976 (request call id 2455) took 3133 ms (client timeout 2879 ms). Trace:
I20260807 01:47:51.287276 30711 consensus_queue.cc:237] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:51.288702 27336 rpcz_store.cc:269] 0807 01:47:48.154827 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:48.154968 (+ 141us) service_pool.cc:224] Handling call
0807 01:47:51.286754 (+3131786us) spinlock_profiling.cc:243] Waited 1.85 ms on lock 0x7b5800003698. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e0521 00007f7d522e03d2 00007f7d5b24c219 00007f7d5a39f640 00007f7d5a467ebb 00007f7d5a467f41 00007f7d5a415eb0 00007f7d5a415794 00007f7d5395e834 00007f7d5395e7c2
0807 01:47:51.288192 (+ 1438us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4264832}
W20260807 01:47:51.294539 27265 debug-util.cc:398] Leaking SignalData structure 0x7b0800130c20 after lost signal to thread 27146
W20260807 01:47:51.294934 27265 debug-util.cc:398] Leaking SignalData structure 0x7b0800339fe0 after lost signal to thread 27148
I20260807 01:47:51.294438 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:51.300292 31102 consensus_queue.cc:1048] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:47:51.305578 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:51.287303 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:51.318425 30778 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:51.323047 27265 debug-util.cc:398] Leaking SignalData structure 0x7b0800100560 after lost signal to thread 27269
W20260807 01:47:51.323508 27265 debug-util.cc:398] Leaking SignalData structure 0x7b0800184680 after lost signal to thread 27270
W20260807 01:47:51.324640 27265 debug-util.cc:398] Leaking SignalData structure 0x7b0800317fa0 after lost signal to thread 27378
W20260807 01:47:51.324935 27265 debug-util.cc:398] Leaking SignalData structure 0x7b0800353020 after lost signal to thread 27379
I20260807 01:47:51.330240 30970 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:47:51.332821 30944 consensus_meta.cc:287] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.524s user 0.012s sys 0.004s
I20260807 01:47:51.333585 30944 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:51.337163 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:51.342960 28715 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Advancing to term 7
W20260807 01:47:51.339690 27265 debug-util.cc:398] Leaking SignalData structure 0x7b080032b020 after lost signal to thread 28748
W20260807 01:47:51.344496 27265 debug-util.cc:398] Leaking SignalData structure 0x7b08003193a0 after lost signal to thread 28750
I20260807 01:47:51.346634 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:51.349212 27289 leader_election.cc:304] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:51.345033 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:51.357161 30778 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:47:51.360298 27265 debug-util.cc:398] Leaking SignalData structure 0x7b0800310a60 after lost signal to thread 31037
I20260807 01:47:51.367241 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:51.370396 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:51.371480 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:51.373926 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:51.374567 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:51.374691 28708 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:51.375730 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:51.376612 27337 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 9.
W20260807 01:47:51.375253 27172 connection.cc:256] RPC call timeout handler was delayed by -1.4953669188544154s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17110000000000003s for a response
I20260807 01:47:51.365178 31135 raft_consensus.cc:2804] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 01:47:51.384723 31135 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:51.385540 31135 raft_consensus.cc:3060] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:51.381728 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:51.359782 31116 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:51.393778 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:47:51.395053 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 1.01662e+06us
I20260807 01:47:51.395298 28710 raft_consensus.cc:1240] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6. Ops: []
I20260807 01:47:51.379984 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:51.396056 31023 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:51.409292 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:51.410562 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:51.416159 28660 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:51.395401 31028 raft_consensus.cc:493] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:51.439677 31116 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:51.440443 31052 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:51.406709 30708 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:51.441062 30944 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:51.441579 31037 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:51.442672 30856 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:51.443377 31024 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:51.412997 30787 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
W20260807 01:47:51.445379 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 46191.6us
I20260807 01:47:51.450954 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:51.452172 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:51.406672 30783 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:51.465852 31102 consensus_queue.cc:1059] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260807 01:47:51.466938 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:51.494535 27325 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:51.495257 27336 raft_consensus.cc:2393] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 7: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:51.468305 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:47:51.496368 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 80303.9us
I20260807 01:47:51.497207 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:51.410027 30941 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:51.449882 30942 raft_consensus.cc:493] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:51.474241 30856 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:51.514194 31102 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:51.514997 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:51.516181 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:51.517411 30778 raft_consensus.cc:3055] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:47:51.463232 31026 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:51.514802 31102 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:51.424537 31023 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:51.517812 30778 raft_consensus.cc:740] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:51.560196 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:51.560501 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:51.561059 27226 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:47:51.561599 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:51.562229 27218 raft_consensus.cc:2468] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 2.
W20260807 01:47:51.564623 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 59600.2us
I20260807 01:47:51.563602 28708 raft_consensus.cc:2468] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:47:51.563578 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:51.569051 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:47:51.573523 27170 connection.cc:256] RPC call timeout handler was delayed by -2.79469034797512s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16200000000000003s for a response
W20260807 01:47:51.574108 27170 connection.cc:256] RPC call timeout handler was delayed by -2.7127903481014073s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17110000000000003s for a response
W20260807 01:47:51.574532 27170 connection.cc:256] RPC call timeout handler was delayed by -2.5678903481457382s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18720000000000003s for a response
I20260807 01:47:51.566921 27229 raft_consensus.cc:2468] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:47:51.580073 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:51.462988 31028 raft_consensus.cc:515] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:51.581202 27213 raft_consensus.cc:2468] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 2.
I20260807 01:47:51.470840 31052 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:51.583499 31052 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:51.584039 31028 leader_election.cc:290] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:51.588912 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:51.566219 30942 raft_consensus.cc:515] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:51.518834 30816 raft_consensus.cc:2804] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:51.604208 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:51.604776 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:51.514070 30787 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:51.602279 27333 raft_consensus.cc:2393] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 4: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:51.607250 27067 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
W20260807 01:47:51.608861 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 68272.1us
I20260807 01:47:51.604923 27222 raft_consensus.cc:2468] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:47:51.472816 31037 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:51.621025 28708 raft_consensus.cc:3060] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Advancing to term 3
I20260807 01:47:51.621923 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:51.624883 27226 raft_consensus.cc:2468] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:47:51.472054 31124 raft_consensus.cc:515] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:51.491220 31024 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:51.470263 30708 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:51.650744 27222 raft_consensus.cc:1275] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:47:51.651873 28715 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
I20260807 01:47:51.518608 30783 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:51.525928 30941 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:51.608992 28707 raft_consensus.cc:1240] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:51.629899 31090 raft_consensus.cc:2804] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:47:51.604550 30816 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:51.673540 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:51.558588 31026 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:51.537883 30928 consensus_queue.cc:1059] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.121s
I20260807 01:47:51.630102 30778 consensus_queue.cc:260] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:51.798780 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 186786us
I20260807 01:47:51.677803 27216 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
I20260807 01:47:51.730502 27326 raft_consensus.cc:1240] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 3. Current term is 4. Ops: []
I20260807 01:47:51.699704 27214 raft_consensus.cc:1275] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 01:47:51.729038 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:51.686270 28708 raft_consensus.cc:1275] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:47:51.949049 27325 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:47:51.649369 31052 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:51.967134 27067 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:51.675988 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:51.964375 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:51.976027 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:51.984892 28710 raft_consensus.cc:1275] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260807 01:47:51.774053 31140 log.cc:927] Time spent T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.175s user 0.000s sys 0.001s
I20260807 01:47:51.984457 27222 raft_consensus.cc:1275] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260807 01:47:51.992027 28660 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
W20260807 01:47:51.978549 27170 connection.cc:256] RPC call timeout handler was delayed by -1.40974878706038s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.264s for a response
I20260807 01:47:51.993533 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:51.993918 27170 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:51.671280 31150 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:51.997362 28660 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:47:51.995554 28655 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:51.699120 28713 raft_consensus.cc:1240] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 6. Current term is 7. Ops: []
I20260807 01:47:51.884913 31024 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:51.774190 30783 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:51.788625 31146 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:51.836539 30970 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:51.836658 30888 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:51.945976 31124 leader_election.cc:290] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:52.050873 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:51.773772 30787 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:51.909853 31037 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:51.884750 30942 leader_election.cc:290] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:51.834822 31113 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:51.876227 31102 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:51.745622 30941 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:52.007560 27170 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:52.007934 27336 raft_consensus.cc:1240] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6. Ops: []
I20260807 01:47:51.969348 30816 raft_consensus.cc:3060] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:52.009532 31167 raft_consensus.cc:2749] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:47:51.770546 31153 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:52.029420 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:52.036419 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:52.010232 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:47:52.037744 27212 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s user 0.044s sys 0.024s
I20260807 01:47:52.030113 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:52.039502 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:51.992394 31148 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:52.093561 27103 raft_consensus.cc:2468] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
W20260807 01:47:52.101115 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 93629.3us
I20260807 01:47:52.046386 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:52.047935 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:51.631747 31007 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:51.996861 30944 raft_consensus.cc:2804] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:47:51.703558 31115 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:52.052630 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:52.052598 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:52.084249 31102 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:52.120194 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:52.054478 30888 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:52.055404 27101 raft_consensus.cc:1240] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:52.039268 31146 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:52.005714 31122 raft_consensus.cc:2804] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:51.746892 31003 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:52.006083 30856 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:52.134805 27330 raft_consensus.cc:2393] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 6: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:52.135264 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.136520 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:51.884249 31135 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:52.138916 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:52.139436 27325 raft_consensus.cc:1275] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:52.068923 27338 raft_consensus.cc:1275] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:47:52.009785 30778 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:51.983844 31052 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:51.965444 31023 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:52.085451 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:52.144237 27336 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:52.144423 27229 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
I20260807 01:47:52.049270 30708 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:52.094657 27102 raft_consensus.cc:2468] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:52.076224 31113 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
W20260807 01:47:52.088759 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 121812us
W20260807 01:47:52.095134 27212 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.567s user 0.044s sys 0.024s
I20260807 01:47:52.045537 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:52.036365 31028 consensus_queue.cc:1048] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260807 01:47:52.070067 30928 raft_consensus.cc:2804] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260807 01:47:52.104947 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 148257us
I20260807 01:47:52.125128 30944 raft_consensus.cc:697] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 7 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:52.106760 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:52.076385 31068 raft_consensus.cc:2804] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:47:52.070402 30681 raft_consensus.cc:493] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:52.071985 30940 raft_consensus.cc:493] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:52.117834 27170 leader_election.cc:304] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:52.175572 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:52.118469 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:52.051558 27222 raft_consensus.cc:2468] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
I20260807 01:47:52.120795 27103 raft_consensus.cc:2468] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:47:52.121099 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:52.121508 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.052258 30970 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:52.123107 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 70199.9us
I20260807 01:47:52.056502 27104 raft_consensus.cc:1275] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:52.107538 30941 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:52.119792 27110 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:52.137902 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.149600 31090 raft_consensus.cc:697] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:52.150210 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.154119 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:47:52.160319 27287 connection.cc:256] RPC call timeout handler was delayed by -1.3854571878910065s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
I20260807 01:47:52.162870 27337 raft_consensus.cc:2393] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 3: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:52.130841 31122 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:52.171552 27172 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:52.217041 27212 raft_consensus.cc:2468] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:47:52.179010 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:47:52.180519 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 62942.7us
I20260807 01:47:52.011514 31150 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:52.117796 31148 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:52.215054 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:52.123286 31053 consensus_queue.cc:1059] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260807 01:47:52.217078 27105 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:47:52.148172 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:52.168458 28706 raft_consensus.cc:1240] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 6. Current term is 7. Ops: []
I20260807 01:47:52.126747 31102 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:52.118924 31153 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:52.240231 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.163262 30928 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:52.246785 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.245445 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.249205 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:47:52.234633 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 63194.7us
I20260807 01:47:52.132627 31167 consensus_queue.cc:1059] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.108s
I20260807 01:47:52.260584 27212 raft_consensus.cc:2468] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 2.
I20260807 01:47:52.183573 31024 raft_consensus.cc:2749] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260807 01:47:52.214336 30787 raft_consensus.cc:2749] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:52.272199 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:52.270576 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:52.278132 27172 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:52.260489 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.278880 27101 raft_consensus.cc:2468] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:47:52.273023 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:52.271000 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:52.241698 31122 raft_consensus.cc:3060] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:52.233214 30783 consensus_queue.cc:1048] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
I20260807 01:47:52.233413 30681 raft_consensus.cc:515] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:52.287485 31167 raft_consensus.cc:3055] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:52.288337 31148 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:52.290295 27172 leader_election.cc:304] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:52.290822 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.243355 31158 raft_consensus.cc:2804] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:47:52.297597 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.299295 30944 consensus_queue.cc:237] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:52.241472 30711 consensus_queue.cc:1048] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260807 01:47:52.240520 30941 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:52.214007 30940 raft_consensus.cc:515] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:52.301110 30970 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:52.301671 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:52.292157 30708 consensus_queue.cc:1059] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260807 01:47:52.304140 31113 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:52.304770 31026 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:52.306427 31115 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:52.307567 30681 leader_election.cc:290] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:52.308619 31146 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:52.309057 31003 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:52.250099 31103 consensus_queue.cc:1048] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
W20260807 01:47:52.310060 31157 log.cc:927] Time spent T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.206s user 0.000s sys 0.000s
I20260807 01:47:52.308645 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:47:52.448851 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 231013us
I20260807 01:47:52.313422 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:52.320353 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:52.322647 31158 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
W20260807 01:47:52.323432 31172 log.cc:927] Time spent T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.080s user 0.001s sys 0.000s
I20260807 01:47:52.323956 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.325695 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:52.256632 31175 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:52.269335 30928 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:47:52.328933 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:47:52.312500 31170 log.cc:927] Time spent T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.145s user 0.000s sys 0.002s
I20260807 01:47:52.404531 31090 consensus_queue.cc:237] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:52.408952 30940 leader_election.cc:290] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:52.409978 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.417785 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:47:52.428480 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 138291us
I20260807 01:47:52.463438 27213 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:47:52.464035 27172 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:52.432814 31052 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:52.464615 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.465333 31102 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:52.441835 31068 raft_consensus.cc:697] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 6 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:52.468230 27175 leader_election.cc:304] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:52.467826 31068 consensus_queue.cc:237] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:52.470839 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.472226 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.465981 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.435787 31135 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:52.438493 30856 raft_consensus.cc:3055] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 6 LEADER]: Stepping down as leader of term 6
W20260807 01:47:52.475893 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 293491us
I20260807 01:47:52.479851 28660 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:47:52.484690 30941 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:52.485342 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:52.486672 27216 raft_consensus.cc:2393] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 7: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:52.439580 27326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36604 (request call id 602) took 1110 ms. Trace:
I20260807 01:47:52.486796 28660 leader_election.cc:304] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:52.487111 27326 rpcz_store.cc:276] 0807 01:47:51.329071 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:51.329204 (+ 133us) service_pool.cc:224] Handling call
0807 01:47:51.329746 (+ 542us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:51.975509 (+645763us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:52.439283 (+463774us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b540012f2d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:47:52.439500 (+ 217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":921114880}
I20260807 01:47:52.315277 31167 raft_consensus.cc:740] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:52.450994 27102 raft_consensus.cc:3060] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:52.488039 28660 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:47:52.449873 27110 raft_consensus.cc:2468] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:47:52.476974 27063 leader_election.cc:304] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
W20260807 01:47:52.453828 27287 connection.cc:256] RPC call timeout handler was delayed by -1.312098914058879s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15380000000000002s for a response
I20260807 01:47:52.455266 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.438386 30787 raft_consensus.cc:2804] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:52.457573 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:47:52.490722 27287 connection.cc:256] RPC call timeout handler was delayed by -1.2444977751001716s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1683s for a response
I20260807 01:47:52.492410 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:52.493201 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:52.494658 30787 raft_consensus.cc:493] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:52.494925 31192 raft_consensus.cc:2804] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:47:52.495596 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.495844 27063 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:52.497162 27063 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:52.497364 31176 raft_consensus.cc:1064] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:52.497480 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.497905 30941 raft_consensus.cc:2804] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:52.498103 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:52.544358 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.511282 27338 raft_consensus.cc:1240] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:52.500491 28660 leader_election.cc:304] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:52.523983 27331 raft_consensus.cc:1275] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:52.548254 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:52.499189 30787 raft_consensus.cc:3060] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:52.549078 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:52.549333 31163 raft_consensus.cc:2804] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:52.549865 31163 raft_consensus.cc:493] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:52.550508 31163 raft_consensus.cc:3060] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:52.550519 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:52.451363 31158 raft_consensus.cc:3060] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:52.509202 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:47:52.581961 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 250789us
I20260807 01:47:52.498540 30941 raft_consensus.cc:493] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:52.498756 27110 raft_consensus.cc:2468] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:47:52.613417 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:52.610603 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.611085 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:52.611311 27214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57662 (request call id 2557) took 1033 ms. Trace:
I20260807 01:47:52.614315 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:52.609987 27064 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:47:52.613679 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:52.614204 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:52.615767 27214 rpcz_store.cc:276] 0807 01:47:51.577845 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:51.577962 (+ 117us) service_pool.cc:224] Handling call
0807 01:47:51.578276 (+ 314us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:51.961493 (+383217us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:52.611020 (+649527us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b5400126954. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:47:52.611234 (+ 214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":267648896}
W20260807 01:47:52.614456 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 117373us
I20260807 01:47:52.617376 27063 leader_election.cc:400] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Vote denied by peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429) with higher term. Message: Invalid argument: T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 3. Current term is 4.
I20260807 01:47:52.617719 27063 leader_election.cc:403] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260807 01:47:52.618858 27063 leader_election.cc:304] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:52.476255 30856 raft_consensus.cc:740] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:52.625844 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:47:52.626647 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 126404us
I20260807 01:47:52.627449 28660 leader_election.cc:304] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:52.620857 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:52.496989 31053 raft_consensus.cc:2749] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:52.622028 27326 raft_consensus.cc:1240] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 1. Current term is 2. Ops: []
I20260807 01:47:52.474890 31007 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:52.623250 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.498801 31052 raft_consensus.cc:2804] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:47:52.624619 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:52.633945 27110 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 9.
I20260807 01:47:52.629554 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.498564 30962 raft_consensus.cc:2749] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260807 01:47:52.499078 31176 raft_consensus.cc:1068] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3: Unable to promote non-voter 57bb4985157447f0a3ab680fa0026f12: Illegal state: Leader has not yet committed an operation in its own term
I20260807 01:47:52.636667 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:52.495505 31192 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:52.633818 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.621074 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:52.635160 27216 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:47:52.630628 27064 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:52.636754 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.642987 28660 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:52.643754 27213 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:47:52.552157 31150 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:52.644249 27064 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:52.650113 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.645090 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:52.650978 27110 raft_consensus.cc:3060] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:52.620218 30941 raft_consensus.cc:3060] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:52.653407 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.620440 31102 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:52.657452 27064 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:52.642995 27222 raft_consensus.cc:2468] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:47:52.659873 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:52.660579 27326 raft_consensus.cc:1275] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260807 01:47:52.634788 30940 raft_consensus.cc:2804] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:52.637549 31052 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:52.665030 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:52.457131 31175 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:52.682780 27064 leader_election.cc:304] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:52.651975 31115 raft_consensus.cc:2804] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:47:52.649523 31192 raft_consensus.cc:3060] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:52.653491 31176 consensus_queue.cc:1048] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260807 01:47:52.707242 31176 raft_consensus.cc:2804] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:52.707643 31176 raft_consensus.cc:493] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:52.708020 31176 raft_consensus.cc:3060] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:52.656970 31103 raft_consensus.cc:2749] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:52.662087 31193 raft_consensus.cc:2804] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:47:52.673785 30940 raft_consensus.cc:493] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
W20260807 01:47:52.704754 30816 consensus_meta.cc:287] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 0.601s user 0.004s sys 0.009s
I20260807 01:47:52.747251 27102 raft_consensus.cc:2468] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:47:52.703251 31115 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:52.687934 31167 consensus_queue.cc:260] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:52.800688 27331 raft_consensus.cc:1240] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6. Ops: []
W20260807 01:47:52.797610 27330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33976 (request call id 2467) took 1570 ms (client timeout 1529 ms). Trace:
I20260807 01:47:52.696436 27101 raft_consensus.cc:2468] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 7.
I20260807 01:47:52.708539 31194 raft_consensus.cc:2749] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 01:47:52.632568 30681 consensus_queue.cc:1048] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260807 01:47:52.679402 31052 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
W20260807 01:47:52.713955 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 94467.8us
I20260807 01:47:52.718641 27216 raft_consensus.cc:1275] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 01:47:52.685859 27287 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:52.722882 31158 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:52.733718 31163 raft_consensus.cc:515] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:52.687289 27326 raft_consensus.cc:1275] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260807 01:47:52.711668 30856 consensus_queue.cc:260] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:52.743582 31184 raft_consensus.cc:2804] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260807 01:47:53.000128 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:52.999538 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:47:52.692864 27064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:53.001051 27103 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:47:52.751323 31173 raft_consensus.cc:2804] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:47:52.803364 31026 raft_consensus.cc:3055] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Stepping down as leader of term 2
W20260807 01:47:52.983857 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 298120us
I20260807 01:47:52.946036 30711 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:52.978744 30962 raft_consensus.cc:2804] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:47:52.984591 27063 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:52.744521 30940 raft_consensus.cc:3060] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Advancing to term 3
W20260807 01:47:52.846127 30778 consensus_meta.cc:287] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.702s user 0.011s sys 0.003s
I20260807 01:47:52.978227 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:52.860997 31003 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:53.001546 31111 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
W20260807 01:47:52.949232 27330 rpcz_store.cc:269] 0807 01:47:51.227471 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:51.227712 (+ 241us) service_pool.cc:224] Handling call
0807 01:47:52.797206 (+1569494us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b5400355cd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:47:52.797406 (+ 200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2081640192}
I20260807 01:47:52.830791 31122 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:53.006363 27216 raft_consensus.cc:1275] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 01:47:52.751204 31197 raft_consensus.cc:2749] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:53.008459 31026 raft_consensus.cc:740] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:52.751538 31193 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:52.835454 31175 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:53.016435 27212 raft_consensus.cc:2393] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 4: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:52.976871 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:52.987192 31007 raft_consensus.cc:2749] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:53.019863 27063 leader_election.cc:304] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:53.022219 31173 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:53.045720 27063 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:53.020560 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:53.022768 27287 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:52.750808 31171 raft_consensus.cc:2804] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:47:53.033332 30856 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:53.028887 27101 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:52.753770 30816 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:52.989614 31115 raft_consensus.cc:3060] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:53.011042 31167 raft_consensus.cc:3060] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:53.005810 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:53.037062 27216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57654 (request call id 1813) took 1036 ms. Trace:
I20260807 01:47:52.751020 31189 raft_consensus.cc:2804] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:47:53.038568 27108 raft_consensus.cc:1240] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6. Ops: []
I20260807 01:47:53.009132 31184 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:53.023896 30711 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:53.026185 31204 raft_consensus.cc:2804] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:47:53.023322 31180 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:53.053439 28708 raft_consensus.cc:1240] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:53.027874 27230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41230 (request call id 403) took 1595 ms. Trace:
I20260807 01:47:53.052501 27063 leader_election.cc:304] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:53.050848 27287 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:53.060489 31171 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:53.062702 27103 raft_consensus.cc:3060] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:53.064730 28711 raft_consensus.cc:2393] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 3: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:47:53.062760 27216 rpcz_store.cc:276] 0807 01:47:52.000920 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:52.983519 (+982599us) service_pool.cc:224] Handling call
0807 01:47:53.005949 (+ 22430us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:53.033261 (+ 27312us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:53.036998 (+ 3737us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:53.043160 27102 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:47:53.042891 31193 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:53.049898 31173 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:53.069821 27287 leader_election.cc:304] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:53.071565 27063 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:53.053023 31135 raft_consensus.cc:2749] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:47:53.066222 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:53.032552 31003 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:53.081024 30711 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:53.068197 27230 rpcz_store.cc:276] 0807 01:47:51.426125 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:51.442770 (+ 16645us) service_pool.cc:224] Handling call
0807 01:47:51.443098 (+ 328us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:51.677228 (+234130us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:51.677261 (+ 33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:51.677302 (+ 41us) raft_consensus.cc:1645] Marking committed up to 9
0807 01:47:51.677341 (+ 39us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:53.021125 (+1343784us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54001f72d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:47:53.021188 (+ 63us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:53.021343 (+ 155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":537904512}
I20260807 01:47:53.054091 31090 raft_consensus.cc:2804] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 01:47:53.121197 31090 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:53.121618 31090 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Advancing to term 10
I20260807 01:47:53.122534 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:53.123895 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:53.074501 31171 raft_consensus.cc:3060] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:53.075573 31180 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:53.036780 31111 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:53.067005 31189 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:53.073951 30681 raft_consensus.cc:2749] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:53.074791 31007 raft_consensus.cc:2804] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:47:53.093452 31026 consensus_queue.cc:260] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:53.125485 31135 raft_consensus.cc:2804] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:53.097716 31163 leader_election.cc:290] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:53.099507 31003 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:53.108624 27287 leader_election.cc:304] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:53.067268 31184 raft_consensus.cc:3060] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:53.120178 31203 raft_consensus.cc:493] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:53.070053 31204 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:53.075657 31208 raft_consensus.cc:2749] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:53.129339 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:53.204928 27336 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:53.133661 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:53.133972 28706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52322 (request call id 617) took 1773 ms. Trace:
I20260807 01:47:53.141083 31117 raft_consensus.cc:3055] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Stepping down as leader of term 3
W20260807 01:47:53.144340 27337 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33976 (request call id 2480) took 1735 ms (client timeout 1670 ms). Trace:
I20260807 01:47:53.145707 27212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57662 (request call id 2579) took 1061 ms. Trace:
I20260807 01:47:53.146271 27333 raft_consensus.cc:1240] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:53.153615 27222 raft_consensus.cc:1240] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 3. Current term is 4. Ops: []
I20260807 01:47:53.160017 28715 raft_consensus.cc:1240] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6. Ops: []
I20260807 01:47:53.213333 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:53.163491 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:53.076146 27063 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
W20260807 01:47:53.466584 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 390499us
I20260807 01:47:53.467669 31208 raft_consensus.cc:2749] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 01:47:53.069734 27064 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
W20260807 01:47:53.172849 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 76837.9us
I20260807 01:47:53.469075 27063 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:47:53.155192 27101 raft_consensus.cc:2468] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:47:53.741559 27101 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 1676) took 1057 ms. Trace:
I20260807 01:47:53.143069 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:53.745348 27099 raft_consensus.cc:3060] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:53.161846 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:53.753372 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:53.119210 27110 raft_consensus.cc:2468] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:47:53.756814 27110 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 1672) took 1124 ms. Trace:
I20260807 01:47:53.757206 27110 rpcz_store.cc:276] 0807 01:47:52.632844 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:52.644848 (+ 12004us) service_pool.cc:224] Handling call
0807 01:47:53.756717 (+1111869us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:53.077797 27100 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:53.884716 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:53.885584 27100 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 1675) took 1211 ms. Trace:
I20260807 01:47:53.305327 27108 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 2896) took 1280 ms. Trace:
I20260807 01:47:53.886519 27108 rpcz_store.cc:276] 0807 01:47:52.024542 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:52.024705 (+ 163us) service_pool.cc:224] Handling call
0807 01:47:52.055142 (+ 30437us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:53.063703 (+1008561us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:53.305059 (+241356us) spinlock_profiling.cc:243] Waited 983 ms on lock 0x7b54003584d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:47:53.305235 (+ 176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2261113600}
I20260807 01:47:53.888013 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:53.889519 27108 raft_consensus.cc:2410] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260807 01:47:53.309229 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:53.373541 27105 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:47:53.429503 26948 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60866 (request call id 216) took 6274 ms. Trace:
I20260807 01:47:53.890900 27104 raft_consensus.cc:2468] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:47:53.892084 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:53.191951 27175 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:53.224236 27226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57662 (request call id 2564) took 1589 ms. Trace:
I20260807 01:47:53.211716 27212 rpcz_store.cc:276] 0807 01:47:52.084598 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:52.454970 (+370372us) service_pool.cc:224] Handling call
0807 01:47:53.145473 (+690503us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b54002639d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:47:53.145643 (+ 170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1210249984}
I20260807 01:47:53.928375 27226 rpcz_store.cc:276] 0807 01:47:51.634624 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:51.656272 (+ 21648us) service_pool.cc:224] Handling call
0807 01:47:52.487030 (+830758us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:52.487226 (+ 196us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:52.487250 (+ 24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:52.487290 (+ 40us) raft_consensus.cc:1645] Marking committed up to 7
0807 01:47:52.487334 (+ 44us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:52.487385 (+ 51us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:53.224018 (+736633us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b54000028d0. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541dc8d 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:47:53.224144 (+ 126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1235706752}
I20260807 01:47:53.930526 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:53.464749 27222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57654 (request call id 1806) took 1491 ms. Trace:
I20260807 01:47:53.932549 27222 rpcz_store.cc:276] 0807 01:47:51.973479 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:52.658982 (+685503us) service_pool.cc:224] Handling call
0807 01:47:53.153433 (+494451us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:53.224533 (+ 71100us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:53.464492 (+239959us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b54002639d0. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541dc8d 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:47:53.464672 (+ 180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1132415744}
W20260807 01:47:53.095361 27229 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57662 (request call id 2568) took 1421 ms (client timeout 1864 ms). Trace:
W20260807 01:47:53.937134 27229 rpcz_store.cc:269] 0807 01:47:51.673490 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:51.675649 (+ 2159us) service_pool.cc:224] Handling call
0807 01:47:53.095127 (+1419478us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b54001de054. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:47:53.095286 (+ 159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":401020672}
I20260807 01:47:53.938735 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:53.940696 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:53.343083 27218 raft_consensus.cc:2468] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:47:53.239785 27227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41230 (request call id 405) took 1786 ms. Trace:
W20260807 01:47:53.203368 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 94829.5us
I20260807 01:47:53.404764 27325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33976 (request call id 2487) took 1428 ms. Trace:
I20260807 01:47:53.303363 27333 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36604 (request call id 613) took 1339 ms. Trace:
I20260807 01:47:53.208731 27328 raft_consensus.cc:1240] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 3. Current term is 4. Ops: []
I20260807 01:47:53.954641 27325 rpcz_store.cc:276] 0807 01:47:51.976550 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:51.976787 (+ 237us) service_pool.cc:224] Handling call
0807 01:47:51.977117 (+ 330us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:52.139823 (+162706us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:53.404421 (+1264598us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b540017b1d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:47:53.404688 (+ 267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":372574592}
I20260807 01:47:53.243755 27336 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
W20260807 01:47:53.211488 27337 rpcz_store.cc:269] 0807 01:47:51.408842 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:51.408993 (+ 151us) service_pool.cc:224] Handling call
0807 01:47:53.144080 (+1735087us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b54002a2954. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:47:53.144258 (+ 178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1713057536}
W20260807 01:47:53.292553 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 89720.2us
I20260807 01:47:53.190667 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:53.332160 28708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48350 (request call id 781) took 1285 ms. Trace:
I20260807 01:47:53.934051 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:53.209244 28706 rpcz_store.cc:276] 0807 01:47:51.360165 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:51.365044 (+ 4879us) service_pool.cc:224] Handling call
0807 01:47:51.504121 (+139077us) spinlock_profiling.cc:243] Waited 3.12 ms on lock 0x7b5800055598. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e0521 00007f7d522e03d2 00007f7d5b24c219 00007f7d5a39f640 00007f7d5a467ebb 00007f7d5a467f41 00007f7d5a429dd0 00007f7d5a414739 00007f7d5a4141e2 00007f7d53959d34
0807 01:47:51.514474 (+ 10353us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:52.255977 (+741503us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:53.133707 (+877730us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b54008831d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:47:53.133902 (+ 195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1452077056}
I20260807 01:47:53.210444 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
W20260807 01:47:53.459820 28711 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52322 (request call id 619) took 1855 ms (client timeout 1501 ms). Trace:
I20260807 01:47:53.419013 28713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48350 (request call id 777) took 1942 ms. Trace:
I20260807 01:47:53.968981 31161 consensus_queue.cc:1059] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:47:53.969013 28713 rpcz_store.cc:276] 0807 01:47:51.476582 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:51.490489 (+ 13907us) service_pool.cc:224] Handling call
0807 01:47:51.658194 (+167705us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:52.011634 (+353440us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:53.418546 (+1406912us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b54008516d0. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541dc8d 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:47:53.418897 (+ 351us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326248192}
I20260807 01:47:53.971223 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:53.461397 28714 raft_consensus.cc:1240] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:53.473444 30681 raft_consensus.cc:2749] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:53.206956 30711 raft_consensus.cc:2804] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:53.178017 30742 consensus_queue.cc:1059] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
W20260807 01:47:53.968148 28711 rpcz_store.cc:269] 0807 01:47:51.604387 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:51.604529 (+ 142us) service_pool.cc:224] Handling call
0807 01:47:53.459534 (+1855005us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b5400107554. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:47:53.459720 (+ 186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3351370496}
I20260807 01:47:53.987972 30742 raft_consensus.cc:3055] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:53.988425 30742 raft_consensus.cc:740] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
W20260807 01:47:53.187884 30787 consensus_meta.cc:287] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.565s user 0.007s sys 0.002s
I20260807 01:47:53.991927 30787 raft_consensus.cc:515] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:53.993140 27213 raft_consensus.cc:1240] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:53.222903 30888 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:47:53.332054 30928 consensus_meta.cc:287] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 0.857s user 0.006s sys 0.006s
I20260807 01:47:54.011835 30928 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:54.013036 28709 raft_consensus.cc:1240] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 8. Current term is 9. Ops: []
I20260807 01:47:53.231904 30940 raft_consensus.cc:515] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:53.273643 30941 consensus_meta.cc:287] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.612s user 0.005s sys 0.008s
I20260807 01:47:53.454751 30962 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:53.401557 31026 raft_consensus.cc:3060] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:53.188752 31007 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:53.154542 31024 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:53.240803 31052 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:53.985220 30711 raft_consensus.cc:493] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:54.014083 30928 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:54.023231 31052 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:53.111917 31053 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:47:53.362665 31102 consensus_meta.cc:287] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.702s user 0.012s sys 0.000s
I20260807 01:47:54.026032 31102 raft_consensus.cc:2749] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429) with higher term. Message: Invalid argument: T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 3. Current term is 4.
I20260807 01:47:53.308483 31090 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:53.251927 31111 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:53.211205 31117 raft_consensus.cc:740] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:53.313536 31115 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:53.544399 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 60796.1us
I20260807 01:47:53.274787 31122 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:47:53.191251 31135 raft_consensus.cc:493] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:54.037418 31135 raft_consensus.cc:3060] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:53.409991 31153 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:54.033757 31117 consensus_queue.cc:260] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:53.124217 31158 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:53.360900 31171 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:53.386274 31173 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:53.242118 31176 consensus_meta.cc:287] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.534s user 0.014s sys 0.000s
I20260807 01:47:54.062647 31176 raft_consensus.cc:515] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:54.063282 31173 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:54.063866 31115 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:54.020848 31024 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:54.064908 31176 leader_election.cc:290] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:53.220945 31180 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:54.024647 31053 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:53.323794 31184 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:54.069974 31184 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:54.062549 31171 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:53.486385 31187 consensus_queue.cc:1048] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:47:53.179976 31189 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
W20260807 01:47:53.237608 31192 consensus_meta.cc:287] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 0.524s user 0.003s sys 0.007s
I20260807 01:47:54.083288 31192 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:54.085732 31192 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:53.296425 31193 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:54.094830 31193 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:53.483508 31194 raft_consensus.cc:2804] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:47:54.096926 31194 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:54.097426 31194 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:54.098363 27216 raft_consensus.cc:1240] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5. Ops: []
I20260807 01:47:54.100389 27216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57654 (request call id 1821) took 2056 ms. Trace:
I20260807 01:47:54.100771 27216 rpcz_store.cc:276] 0807 01:47:52.043642 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:53.077396 (+1033754us) service_pool.cc:224] Handling call
0807 01:47:53.130714 (+ 53318us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.098869 (+968155us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.100098 (+ 1229us) spinlock_profiling.cc:243] Waited 967 ms on lock 0x7b54001dea54. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:47:54.100317 (+ 219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2224916864}
I20260807 01:47:53.227475 31203 raft_consensus.cc:515] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:54.101913 31135 raft_consensus.cc:515] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:54.103441 27230 raft_consensus.cc:2468] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:47:54.104591 31203 leader_election.cc:290] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:53.207026 31204 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:54.106648 27230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57654 (request call id 1845) took 1010 ms. Trace:
I20260807 01:47:53.209781 31207 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:54.111193 31207 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:54.113741 31207 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:53.469112 31208 consensus_queue.cc:1048] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:47:53.624941 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:54.115408 28710 raft_consensus.cc:2393] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 7: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
W20260807 01:47:53.674801 27103 consensus_meta.cc:287] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.602s user 0.013s sys 0.004s
I20260807 01:47:54.116271 27103 raft_consensus.cc:2468] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
W20260807 01:47:54.116757 27103 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 1667) took 1660 ms (client timeout 2093 ms). Trace:
W20260807 01:47:54.117101 27103 rpcz_store.cc:269] 0807 01:47:52.456487 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:53.003433 (+546946us) service_pool.cc:224] Handling call
0807 01:47:54.116695 (+1113262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.118058 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:54.118810 27103 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41080 (request call id 2914) took 1100 ms. Trace:
I20260807 01:47:54.119094 27103 rpcz_store.cc:276] 0807 01:47:53.018479 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.117795 (+1099316us) service_pool.cc:224] Handling call
0807 01:47:54.118731 (+ 936us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.119484 30778 raft_consensus.cc:1064] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
W20260807 01:47:54.119837 30778 raft_consensus.cc:1068] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3: Unable to promote non-voter 57bb4985157447f0a3ab680fa0026f12: Illegal state: Leader has not yet committed an operation in its own term
I20260807 01:47:54.119975 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:54.120572 27103 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:47:53.679188 31150 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:54.122581 31150 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:54.124644 31150 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:53.679961 31201 raft_consensus.cc:493] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:54.125298 31201 raft_consensus.cc:515] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:54.107019 27230 rpcz_store.cc:276] 0807 01:47:53.095980 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:53.133364 (+ 37384us) service_pool.cc:224] Handling call
0807 01:47:54.106467 (+973103us) spinlock_profiling.cc:243] Waited 897 ms on lock 0x7b54000aa854. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:47:54.106583 (+ 116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2063762816}
I20260807 01:47:54.126936 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:54.127338 31201 leader_election.cc:290] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:53.682972 27170 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:53.707324 27331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59490 (request call id 2488) took 1166 ms. Trace:
I20260807 01:47:53.725419 30942 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:53.761756 31175 raft_consensus.cc:2749] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:53.764818 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:54.128127 27331 rpcz_store.cc:276] 0807 01:47:52.540639 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:52.626788 (+ 86149us) service_pool.cc:224] Handling call
0807 01:47:52.800441 (+173653us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:52.944419 (+143978us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:53.707040 (+762621us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b5400355cd0. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541dc8d 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:47:53.707238 (+ 198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398562688}
I20260807 01:47:54.128778 27110 raft_consensus.cc:2410] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260807 01:47:53.765807 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:53.741942 27101 rpcz_store.cc:276] 0807 01:47:52.684119 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:52.977901 (+293782us) service_pool.cc:224] Handling call
0807 01:47:53.741484 (+763583us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.130646 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:47:54.131148 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 448265us
I20260807 01:47:54.131508 27101 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41080 (request call id 2915) took 1102 ms. Trace:
I20260807 01:47:54.131875 27101 rpcz_store.cc:276] 0807 01:47:53.028679 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.130304 (+1101625us) service_pool.cc:224] Handling call
0807 01:47:54.131449 (+ 1145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.132483 27170 leader_election.cc:304] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:53.769543 27099 raft_consensus.cc:2468] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:47:53.776880 27109 raft_consensus.cc:1240] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 7 LEADER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 6. Current term is 7. Ops: []
I20260807 01:47:54.134357 31207 raft_consensus.cc:2749] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:54.134846 31150 consensus_queue.cc:1048] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:47:54.135174 31207 raft_consensus.cc:2749] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:54.135890 27109 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 2893) took 2123 ms. Trace:
I20260807 01:47:54.136279 27109 rpcz_store.cc:276] 0807 01:47:52.012328 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:52.022374 (+ 10046us) service_pool.cc:224] Handling call
0807 01:47:52.057569 (+ 35195us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.134828 (+2077259us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.135648 (+ 820us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b54000b18d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:47:54.135830 (+ 182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3952187136}
I20260807 01:47:54.137212 27170 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:53.892170 31103 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:54.138523 31201 raft_consensus.cc:2749] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:54.138446 31103 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:54.129998 27103 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41080 (request call id 2909) took 1125 ms. Trace:
I20260807 01:47:54.139500 27103 rpcz_store.cc:276] 0807 01:47:53.005463 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.119721 (+1114258us) service_pool.cc:224] Handling call
0807 01:47:54.129935 (+ 10214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.141026 27103 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1654) took 2003 ms. Trace:
I20260807 01:47:54.140741 31189 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:53.472903 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 403196us
I20260807 01:47:54.142206 27064 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:54.143945 27064 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:54.144394 31150 consensus_queue.cc:1059] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260807 01:47:53.896793 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:52.545504 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:54.145959 31201 raft_consensus.cc:3055] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Stepping down as leader of term 1
I20260807 01:47:53.891004 26948 rpcz_store.cc:276] 0807 01:47:47.154920 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:47.175932 (+ 21012us) service_pool.cc:224] Handling call
0807 01:47:53.429315 (+6253383us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 01:47:51.219340 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 01:47:51.225018 (+ 5678us) write_op.cc:432] Acquiring schema lock in shared mode
0807 01:47:51.225052 (+ 34us) write_op.cc:435] Acquired schema lock
0807 01:47:51.225079 (+ 27us) tablet.cc:662] Decoding operations
0807 01:47:51.225487 (+ 408us) write_op.cc:620] Acquiring the partition lock for write op
0807 01:47:51.225577 (+ 90us) write_op.cc:641] Partition lock acquired for write op
0807 01:47:51.225605 (+ 28us) tablet.cc:685] Acquiring locks for 2 operations
0807 01:47:51.225964 (+ 359us) tablet.cc:701] Row locks acquired
0807 01:47:51.225977 (+ 13us) write_op.cc:260] PREPARE: finished
0807 01:47:51.230621 (+ 4644us) write_op.cc:270] Start()
0807 01:47:51.230787 (+ 166us) write_op.cc:276] Timestamp: P: 1786067271230491 usec, L: 0
0807 01:47:51.230806 (+ 19us) op_driver.cc:348] REPLICATION: starting
0807 01:47:51.233740 (+ 2934us) log.cc:844] Serialized 901 byte log entry
0807 01:47:51.596764 (+363024us) op_driver.cc:464] REPLICATION: finished
0807 01:47:52.131635 (+534871us) write_op.cc:301] APPLY: starting
0807 01:47:52.142330 (+ 10695us) tablet.cc:1367] starting BulkCheckPresence
0807 01:47:52.169827 (+ 27497us) tablet.cc:1370] finished BulkCheckPresence
0807 01:47:52.244070 (+ 74243us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 01:47:52.326497 (+ 82427us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 01:47:52.326793 (+ 296us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0807 01:47:52.338729 (+ 11936us) write_op.cc:312] APPLY: finished
0807 01:47:52.441811 (+103082us) log.cc:844] Serialized 29 byte log entry
0807 01:47:52.457845 (+ 16034us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:47:52.462573 (+ 4728us) write_op.cc:454] Released schema lock
0807 01:47:52.475440 (+ 12867us) write_op.cc:341] FINISH: Updating metrics
0807 01:47:52.476442 (+ 1002us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:47:52.476476 (+ 34us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":523011,"apply.run_cpu_time_us":3876,"apply.run_wall_time_us":352792,"num_ops":2,"prepare.queue_time_us":407797,"prepare.run_cpu_time_us":15570,"prepare.run_wall_time_us":156121,"raft.queue_time_us":210528,"raft.run_cpu_time_us":12913,"raft.run_wall_time_us":411791,"replication_time_us":365693,"thread_start_us":773777,"threads_started":4,"wal-append.queue_time_us":144878,"wal-append.run_cpu_time_us":20023,"wal-append.run_wall_time_us":2097633}]]}
I20260807 01:47:53.885931 27100 rpcz_store.cc:276] 0807 01:47:52.674328 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:53.884443 (+1210115us) service_pool.cc:224] Handling call
0807 01:47:53.885502 (+ 1059us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:53.908583 27108 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 1662) took 1656 ms. Trace:
I20260807 01:47:54.148907 27108 rpcz_store.cc:276] 0807 01:47:52.233796 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:53.887718 (+1653922us) service_pool.cc:224] Handling call
0807 01:47:53.890087 (+ 2369us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 01:47:54.134433 27099 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 1673) took 1495 ms (client timeout 1913 ms). Trace:
W20260807 01:47:54.149619 27099 rpcz_store.cc:269] 0807 01:47:52.639405 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:53.142792 (+503387us) service_pool.cc:224] Handling call
0807 01:47:54.134372 (+991580us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.150334 27108 raft_consensus.cc:1275] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260807 01:47:54.150755 27099 raft_consensus.cc:1240] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 9. Current term is 10. Ops: []
I20260807 01:47:54.150956 27108 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1659) took 1987 ms. Trace:
I20260807 01:47:54.151327 27108 rpcz_store.cc:276] 0807 01:47:52.163828 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.149682 (+1985854us) service_pool.cc:224] Handling call
0807 01:47:54.149997 (+ 315us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.150695 (+ 698us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.150897 (+ 202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.152527 27108 raft_consensus.cc:1240] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:54.153075 27108 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1661) took 1973 ms. Trace:
I20260807 01:47:54.153364 27108 rpcz_store.cc:276] 0807 01:47:52.179471 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.152054 (+1972583us) service_pool.cc:224] Handling call
0807 01:47:54.152366 (+ 312us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.152859 (+ 493us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.153023 (+ 164us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 01:47:53.909204 27105 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52556 (request call id 544) took 1739 ms (client timeout 1676 ms). Trace:
I20260807 01:47:53.910373 27214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57654 (request call id 1808) took 1936 ms. Trace:
I20260807 01:47:53.914853 30854 raft_consensus.cc:2749] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260807 01:47:54.154326 27105 rpcz_store.cc:269] 0807 01:47:52.170124 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:52.248929 (+ 78805us) service_pool.cc:224] Handling call
0807 01:47:53.908981 (+1660052us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b5400031454. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:47:53.909123 (+ 142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1772234624}
I20260807 01:47:54.149024 30942 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:53.920957 30816 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:47:53.923524 28660 leader_election.cc:336] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Timed out: RequestConsensusVote RPC to 127.24.4.1:35499 timed out after 1.536s (SENT)
I20260807 01:47:54.156635 28660 leader_election.cc:304] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:54.157152 30942 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:47:54.157496 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 234043us
I20260807 01:47:54.158558 31192 raft_consensus.cc:2749] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:53.931655 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:54.159215 31192 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:54.159718 27226 raft_consensus.cc:2468] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:47:54.159821 31192 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:53.945253 27227 rpcz_store.cc:276] 0807 01:47:51.452924 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:51.453046 (+ 122us) service_pool.cc:224] Handling call
0807 01:47:51.466528 (+ 13482us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:51.584498 (+117970us) raft_consensus.cc:1534] Early marking committed up to index 6
0807 01:47:51.649158 (+ 64660us) log.cc:844] Serialized 14 byte log entry
0807 01:47:52.058702 (+409544us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:52.058767 (+ 65us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:47:52.058821 (+ 54us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:53.239510 (+1180689us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54001fc054. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:47:53.239565 (+ 55us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:53.239712 (+ 147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16306,"prepare.queue_time_us":155234,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":72,"raft.queue_time_us":20130,"raft.run_cpu_time_us":538,"raft.run_wall_time_us":19797,"spinlock_wait_cycles":270554880,"thread_start_us":459319,"threads_started":2,"wal-append.queue_time_us":505514,"wal-append.run_cpu_time_us":3218,"wal-append.run_wall_time_us":1295652}
I20260807 01:47:53.953661 27287 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:54.160485 31208 raft_consensus.cc:2749] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260807 01:47:54.160988 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 207461us
I20260807 01:47:54.161625 31180 raft_consensus.cc:2804] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 01:47:54.161710 27287 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:54.162253 31180 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:54.162652 31180 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Advancing to term 10
I20260807 01:47:53.944530 27218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41230 (request call id 418) took 1297 ms. Trace:
I20260807 01:47:53.955315 27333 rpcz_store.cc:276] 0807 01:47:51.963574 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:51.963752 (+ 178us) service_pool.cc:224] Handling call
0807 01:47:53.146059 (+1182307us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:53.211953 (+ 65894us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:53.303089 (+ 91136us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54002a2950. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541dc8d 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:47:53.303285 (+ 196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2718237952}
I20260807 01:47:53.966471 31213 raft_consensus.cc:493] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:54.164173 31213 raft_consensus.cc:515] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:54.166337 31213 leader_election.cc:290] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:53.967247 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:54.167848 27222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57654 (request call id 1848) took 1071 ms. Trace:
I20260807 01:47:54.168184 27222 rpcz_store.cc:276] 0807 01:47:53.096743 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:53.965573 (+868830us) service_pool.cc:224] Handling call
0807 01:47:54.167776 (+202203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":543488}
I20260807 01:47:54.169215 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:54.157634 27214 rpcz_store.cc:276] 0807 01:47:51.973961 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:52.662144 (+688183us) service_pool.cc:224] Handling call
0807 01:47:53.096509 (+434365us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:47:53.122372 (+ 25863us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:53.122402 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:47:53.122975 (+ 573us) log.cc:844] Serialized 26 byte log entry
0807 01:47:53.230948 (+107973us) raft_consensus.cc:1645] Marking committed up to 3
0807 01:47:53.230978 (+ 30us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:53.231047 (+ 69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:47:53.325137 (+ 94090us) raft_consensus.cc:1685] finished
0807 01:47:53.325163 (+ 26us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:53.910057 (+584894us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b54001de050. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541dc8d 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:47:53.910308 (+ 251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31302,"raft.queue_time_us":54115,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":19630,"spinlock_wait_cycles":998056832,"thread_start_us":107596,"threads_started":1,"wal-append.queue_time_us":161777}
I20260807 01:47:54.129321 27110 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41080 (request call id 2910) took 1123 ms. Trace:
I20260807 01:47:53.931581 27212 raft_consensus.cc:2393] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 7: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:54.170296 27110 rpcz_store.cc:276] 0807 01:47:53.005763 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:53.758086 (+752323us) service_pool.cc:224] Handling call
0807 01:47:54.129233 (+371147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.170965 27212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57654 (request call id 1842) took 1091 ms. Trace:
I20260807 01:47:54.169297 31192 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:53.968257 28655 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
W20260807 01:47:54.171864 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 203761us
I20260807 01:47:54.171981 27110 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1671) took 1547 ms. Trace:
I20260807 01:47:53.964407 28708 rpcz_store.cc:276] 0807 01:47:52.046852 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:52.047002 (+ 150us) service_pool.cc:224] Handling call
0807 01:47:52.047309 (+ 307us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:53.067927 (+1020618us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:53.331878 (+263951us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54004b7154. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:47:53.332085 (+ 207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2313443712}
I20260807 01:47:53.966440 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:54.172441 27110 rpcz_store.cc:276] 0807 01:47:52.624665 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.171118 (+1546453us) service_pool.cc:224] Handling call
0807 01:47:54.171466 (+ 348us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.171646 (+ 180us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:54.171676 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:54.171723 (+ 47us) raft_consensus.cc:1645] Marking committed up to 4
0807 01:47:54.171762 (+ 39us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.171817 (+ 55us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:54.171922 (+ 105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:53.999074 30787 leader_election.cc:290] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:54.174252 27110 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1677) took 1467 ms. Trace:
I20260807 01:47:54.001391 27213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57654 (request call id 1809) took 2027 ms. Trace:
I20260807 01:47:53.968377 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:54.174612 27110 rpcz_store.cc:276] 0807 01:47:52.706465 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.173456 (+1466991us) service_pool.cc:224] Handling call
0807 01:47:54.173753 (+ 297us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.173921 (+ 168us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:54.173944 (+ 23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:54.173981 (+ 37us) raft_consensus.cc:1645] Marking committed up to 2
0807 01:47:54.174015 (+ 34us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.174066 (+ 51us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:54.174200 (+ 134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.175355 27229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57654 (request call id 1840) took 1096 ms. Trace:
I20260807 01:47:54.175668 27229 rpcz_store.cc:276] 0807 01:47:53.078858 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:53.941952 (+863094us) service_pool.cc:224] Handling call
0807 01:47:54.175297 (+233345us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.176544 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:54.177096 27229 raft_consensus.cc:3060] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:54.006469 28714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48350 (request call id 780) took 1960 ms. Trace:
I20260807 01:47:54.016527 30941 raft_consensus.cc:515] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:54.004712 30742 consensus_queue.cc:260] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:54.178372 27218 rpcz_store.cc:276] 0807 01:47:52.646892 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:52.659585 (+ 12693us) service_pool.cc:224] Handling call
0807 01:47:53.944231 (+1284646us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b54000f2b54. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:47:53.944426 (+ 195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1569149184}
I20260807 01:47:54.179299 30742 raft_consensus.cc:3060] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:54.179589 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:54.179901 30941 leader_election.cc:290] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:54.043792 27104 raft_consensus.cc:2468] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:47:54.020095 31007 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:54.017478 30962 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:47:54.186204 27331 raft_consensus.cc:1275] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260807 01:47:54.054143 31117 raft_consensus.cc:3060] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:54.187026 27214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57662 (request call id 2580) took 2063 ms. Trace:
I20260807 01:47:54.022104 31003 consensus_queue.cc:1059] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260807 01:47:54.016695 30940 leader_election.cc:290] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:54.058878 28715 raft_consensus.cc:1240] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5. Ops: []
I20260807 01:47:54.184619 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:54.148275 27100 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1658) took 1992 ms. Trace:
I20260807 01:47:54.184891 30854 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:54.189545 28715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48362 (request call id 733) took 1027 ms. Trace:
W20260807 01:47:54.189759 27172 leader_election.cc:336] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Timed out: RequestConsensusVote RPC to 127.24.4.1:35499 timed out after 1.503s (SENT)
I20260807 01:47:54.047343 31090 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:54.067831 31053 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:54.086403 28707 raft_consensus.cc:1240] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 3. Current term is 4. Ops: []
I20260807 01:47:54.066051 31024 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:54.102130 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:54.191128 27218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57654 (request call id 1823) took 2064 ms. Trace:
I20260807 01:47:54.191251 27216 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:47:54.104871 31135 leader_election.cc:290] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:54.105480 27326 raft_consensus.cc:1240] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:54.195466 28715 rpcz_store.cc:276] 0807 01:47:53.178675 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:53.225338 (+ 46663us) service_pool.cc:224] Handling call
0807 01:47:53.235802 (+ 10464us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.188048 (+952246us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.189322 (+ 1274us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b54003eb554. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:47:54.189485 (+ 163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1892492672}
I20260807 01:47:54.198211 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:54.192354 27172 leader_election.cc:304] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:54.121155 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:54.200371 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:53.678318 31164 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:54.127526 27230 raft_consensus.cc:2468] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:47:53.899189 26950 catalog_manager.cc:5697] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:54.148250 31175 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
W20260807 01:47:53.913957 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 722066us
I20260807 01:47:54.160257 27226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57654 (request call id 1843) took 1080 ms. Trace:
I20260807 01:47:54.163363 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:54.022531 30711 raft_consensus.cc:3060] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:54.182251 27104 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 1665) took 1897 ms. Trace:
I20260807 01:47:54.183815 27099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 2907) took 2011 ms. Trace:
I20260807 01:47:54.246383 27099 rpcz_store.cc:276] 0807 01:47:52.172544 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.150316 (+1977772us) service_pool.cc:224] Handling call
0807 01:47:54.150577 (+ 261us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.183580 (+ 33003us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.183761 (+ 181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.248282 27099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 549) took 1244 ms. Trace:
I20260807 01:47:54.248646 27099 rpcz_store.cc:276] 0807 01:47:53.003781 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.247330 (+1243549us) service_pool.cc:224] Handling call
0807 01:47:54.247709 (+ 379us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.247916 (+ 207us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:54.247941 (+ 25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:54.247987 (+ 46us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:47:54.248026 (+ 39us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.248081 (+ 55us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:54.248214 (+ 133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.171361 27212 rpcz_store.cc:276] 0807 01:47:53.079369 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:53.930113 (+850744us) service_pool.cc:224] Handling call
0807 01:47:54.170893 (+240780us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.178339 28714 rpcz_store.cc:276] 0807 01:47:52.046190 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:52.051871 (+ 5681us) service_pool.cc:224] Handling call
0807 01:47:53.461187 (+1409316us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:53.979380 (+518193us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.006223 (+ 26843us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400107550. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541dc8d 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:47:54.006395 (+ 172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3239294720}
I20260807 01:47:54.137945 27109 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1652) took 2001 ms. Trace:
I20260807 01:47:54.187390 27214 rpcz_store.cc:276] 0807 01:47:52.124116 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.186274 (+2062158us) service_pool.cc:224] Handling call
0807 01:47:54.186543 (+ 269us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.186721 (+ 178us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:54.186744 (+ 23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:54.186782 (+ 38us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:47:54.186815 (+ 33us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.186867 (+ 52us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:54.186971 (+ 104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.145488 27064 leader_election.cc:304] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:47:54.191744 27216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57654 (request call id 1849) took 1094 ms. Trace:
I20260807 01:47:54.182253 27213 rpcz_store.cc:276] 0807 01:47:51.999754 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:52.681604 (+681850us) service_pool.cc:224] Handling call
0807 01:47:52.710402 (+ 28798us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:53.993582 (+1283180us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.001108 (+ 7526us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54001df454. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:47:54.001322 (+ 214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2949625216}
I20260807 01:47:54.169853 27222 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
W20260807 01:47:54.159126 28660 leader_election.cc:336] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Timed out: RequestConsensusVote RPC to 127.24.4.1:35499 timed out after 1.643s (SENT)
I20260807 01:47:54.188985 27100 rpcz_store.cc:276] 0807 01:47:52.155632 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.147390 (+1991758us) service_pool.cc:224] Handling call
0807 01:47:54.147719 (+ 329us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.147918 (+ 199us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:54.147942 (+ 24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:54.147986 (+ 44us) raft_consensus.cc:1645] Marking committed up to 3
0807 01:47:54.148033 (+ 47us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.148088 (+ 55us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:54.148210 (+ 122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.201355 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:54.172746 31192 raft_consensus.cc:2804] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:47:54.191497 27218 rpcz_store.cc:276] 0807 01:47:52.126502 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.190325 (+2063823us) service_pool.cc:224] Handling call
0807 01:47:54.190607 (+ 282us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.190810 (+ 203us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:54.190833 (+ 23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:54.190875 (+ 42us) raft_consensus.cc:1645] Marking committed up to 2
0807 01:47:54.190930 (+ 55us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.190980 (+ 50us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:54.191078 (+ 98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.204443 26950 catalog_manager.cc:5697] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:54.205771 27230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57654 (request call id 1850) took 1108 ms. Trace:
I20260807 01:47:54.206317 31201 raft_consensus.cc:740] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:54.212713 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:54.213825 28706 raft_consensus.cc:1240] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6. Ops: []
I20260807 01:47:54.214008 28709 raft_consensus.cc:1240] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:54.141585 31103 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:47:54.216885 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 55244us
I20260807 01:47:54.217331 31231 raft_consensus.cc:2804] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:54.202841 31090 consensus_queue.cc:1059] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260807 01:47:54.218827 27108 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1663) took 1975 ms. Trace:
I20260807 01:47:54.287478 27108 rpcz_store.cc:276] 0807 01:47:52.243558 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.154090 (+1910532us) service_pool.cc:224] Handling call
0807 01:47:54.218262 (+ 64172us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.218472 (+ 210us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:54.218498 (+ 26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:54.218542 (+ 44us) raft_consensus.cc:1645] Marking committed up to 3
0807 01:47:54.218581 (+ 39us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.218634 (+ 53us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:54.218759 (+ 125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.287644 28711 raft_consensus.cc:1240] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:54.289289 27108 raft_consensus.cc:1240] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 3. Current term is 4. Ops: []
I20260807 01:47:54.189781 30854 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:54.293269 30854 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:54.145581 31150 consensus_queue.cc:1048] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260807 01:47:54.220646 31164 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:54.221354 31024 raft_consensus.cc:2749] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:54.141397 27103 rpcz_store.cc:276] 0807 01:47:52.137415 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.140206 (+2002791us) service_pool.cc:224] Handling call
0807 01:47:54.140510 (+ 304us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.140700 (+ 190us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:54.140726 (+ 26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:54.140765 (+ 39us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:47:54.140798 (+ 33us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.140851 (+ 53us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:54.140967 (+ 116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.225677 27326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36604 (request call id 616) took 1099 ms. Trace:
I20260807 01:47:54.307126 27326 rpcz_store.cc:276] 0807 01:47:53.126338 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:53.126473 (+ 135us) service_pool.cc:224] Handling call
0807 01:47:53.181526 (+ 55053us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.197566 (+1016040us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.225411 (+ 27845us) spinlock_profiling.cc:243] Waited 924 ms on lock 0x7b540027c9d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:47:54.225616 (+ 205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2124413696}
W20260807 01:47:54.224184 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 80026.2us
I20260807 01:47:54.225040 27227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57654 (request call id 1851) took 1117 ms. Trace:
I20260807 01:47:54.309618 27227 rpcz_store.cc:276] 0807 01:47:53.107649 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.162950 (+1055301us) service_pool.cc:224] Handling call
0807 01:47:54.224971 (+ 62021us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.224498 27226 rpcz_store.cc:276] 0807 01:47:53.086816 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:53.930113 (+843297us) service_pool.cc:224] Handling call
0807 01:47:54.160191 (+230078us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.224792 27175 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:54.251878 27109 rpcz_store.cc:276] 0807 01:47:52.136789 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.137158 (+2000369us) service_pool.cc:224] Handling call
0807 01:47:54.137457 (+ 299us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.137632 (+ 175us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:54.137654 (+ 22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:54.137696 (+ 42us) raft_consensus.cc:1645] Marking committed up to 5
0807 01:47:54.137745 (+ 49us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.137794 (+ 49us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:54.137892 (+ 98us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 01:47:54.253084 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 107645us
I20260807 01:47:54.253140 28660 leader_election.cc:304] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
W20260807 01:47:54.260684 28662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:54.311600 27064 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:47:54.257252 27230 rpcz_store.cc:276] 0807 01:47:53.107291 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.126678 (+1019387us) service_pool.cc:224] Handling call
0807 01:47:54.205703 (+ 79025us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.252583 27216 rpcz_store.cc:276] 0807 01:47:53.096976 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.101749 (+1004773us) service_pool.cc:224] Handling call
0807 01:47:54.191691 (+ 89942us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.246111 27104 rpcz_store.cc:276] 0807 01:47:52.284752 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:53.891777 (+1607025us) service_pool.cc:224] Handling call
0807 01:47:54.182056 (+290279us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b54003368d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:47:54.182187 (+ 131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":309162368}
I20260807 01:47:54.250114 27099 raft_consensus.cc:1240] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:54.253100 27222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57654 (request call id 1839) took 1174 ms. Trace:
I20260807 01:47:54.313308 27222 rpcz_store.cc:276] 0807 01:47:53.078486 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.168953 (+1090467us) service_pool.cc:224] Handling call
0807 01:47:54.253039 (+ 84086us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.255939 27218 raft_consensus.cc:1275] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260807 01:47:54.264544 28708 raft_consensus.cc:2393] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 3: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:47:54.266405 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:54.268074 27212 raft_consensus.cc:1275] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:54.257103 26950 catalog_manager.cc:5697] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } }
I20260807 01:47:54.215745 31189 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:54.270884 27100 raft_consensus.cc:1240] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:54.314558 27218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57654 (request call id 1852) took 1206 ms. Trace:
I20260807 01:47:54.268965 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:54.283485 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:54.286742 27287 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:54.316640 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:54.316937 27099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 550) took 1312 ms. Trace:
I20260807 01:47:54.289949 27108 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 553) took 1285 ms. Trace:
I20260807 01:47:54.315398 27212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57662 (request call id 2581) took 2145 ms. Trace:
W20260807 01:47:54.324172 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 99498.8us
I20260807 01:47:54.317585 27213 raft_consensus.cc:3055] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:47:54.326498 27099 rpcz_store.cc:276] 0807 01:47:53.004013 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.249582 (+1245569us) service_pool.cc:224] Handling call
0807 01:47:54.249930 (+ 348us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.313284 (+ 63354us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.316863 (+ 3579us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.327499 27213 raft_consensus.cc:740] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:54.327023 27064 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:54.328238 27099 raft_consensus.cc:1240] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5. Ops: []
W20260807 01:47:54.321007 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 161908us
W20260807 01:47:54.329550 28660 leader_election.cc:336] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Timed out: RequestConsensusVote RPC to 127.24.4.1:35499 timed out after 1.643s (SENT)
I20260807 01:47:54.330094 28660 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:54.299271 31207 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:54.333089 27175 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:54.270581 31213 raft_consensus.cc:2804] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:47:54.326778 27108 rpcz_store.cc:276] 0807 01:47:53.004556 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.288723 (+1284167us) service_pool.cc:224] Handling call
0807 01:47:54.289105 (+ 382us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.289721 (+ 616us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.289893 (+ 172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.327297 27212 rpcz_store.cc:276] 0807 01:47:52.169655 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.250828 (+2081173us) service_pool.cc:224] Handling call
0807 01:47:54.251170 (+ 342us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.315169 (+ 63999us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.315344 (+ 175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.331317 27103 raft_consensus.cc:1240] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6. Ops: []
I20260807 01:47:54.337067 27103 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 2911) took 1331 ms. Trace:
I20260807 01:47:54.337459 27103 rpcz_store.cc:276] 0807 01:47:53.006020 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.330089 (+1324069us) service_pool.cc:224] Handling call
0807 01:47:54.331156 (+ 1067us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.336806 (+ 5650us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.337009 (+ 203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.339094 27103 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 2918) took 1282 ms. Trace:
I20260807 01:47:54.339449 27103 rpcz_store.cc:276] 0807 01:47:53.057201 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.338220 (+1281019us) service_pool.cc:224] Handling call
0807 01:47:54.338497 (+ 277us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.338764 (+ 267us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:54.338792 (+ 28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:54.338837 (+ 45us) raft_consensus.cc:1645] Marking committed up to 5
0807 01:47:54.338873 (+ 36us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.338933 (+ 60us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:54.339038 (+ 105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.341045 27103 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 2919) took 1272 ms. Trace:
I20260807 01:47:54.341357 27103 rpcz_store.cc:276] 0807 01:47:53.068680 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.340316 (+1271636us) service_pool.cc:224] Handling call
0807 01:47:54.340577 (+ 261us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.340745 (+ 168us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:54.340769 (+ 24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:54.340808 (+ 39us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:47:54.340846 (+ 38us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.340897 (+ 51us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:54.340995 (+ 98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.318500 27100 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 552) took 1314 ms. Trace:
I20260807 01:47:54.331449 27064 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:47:54.342320 27100 rpcz_store.cc:276] 0807 01:47:53.004398 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.253623 (+1249225us) service_pool.cc:224] Handling call
0807 01:47:54.253969 (+ 346us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.318251 (+ 64282us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.318444 (+ 193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.318353 27218 rpcz_store.cc:276] 0807 01:47:53.107934 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.255241 (+1147307us) service_pool.cc:224] Handling call
0807 01:47:54.255563 (+ 322us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.314266 (+ 58703us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.314498 (+ 232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.333333 27099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1678) took 1298 ms. Trace:
I20260807 01:47:54.344399 27099 rpcz_store.cc:276] 0807 01:47:53.035289 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.327795 (+1292506us) service_pool.cc:224] Handling call
0807 01:47:54.328064 (+ 269us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.328643 (+ 579us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.333262 (+ 4619us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.329617 27104 raft_consensus.cc:1240] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6. Ops: []
I20260807 01:47:54.346189 27104 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 2916) took 1310 ms. Trace:
I20260807 01:47:54.346525 27104 rpcz_store.cc:276] 0807 01:47:53.035828 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.329171 (+1293343us) service_pool.cc:224] Handling call
0807 01:47:54.329442 (+ 271us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.345895 (+ 16453us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.346097 (+ 202us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 01:47:54.349486 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 149017us
W20260807 01:47:54.351101 27172 leader_election.cc:336] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Timed out: RequestConsensusVote RPC to 127.24.4.1:35499 timed out after 1.711s (SENT)
I20260807 01:47:54.351526 27172 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:54.197350 31204 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:54.358487 31204 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:54.299597 31192 raft_consensus.cc:697] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 7 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:54.316277 28707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48362 (request call id 738) took 1078 ms. Trace:
I20260807 01:47:54.370272 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:54.380030 27099 raft_consensus.cc:1240] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260807 01:47:54.380887 27100 raft_consensus.cc:1240] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 3. Current term is 4. Ops: []
I20260807 01:47:54.306442 31231 raft_consensus.cc:697] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:54.385998 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:54.359289 30854 raft_consensus.cc:2749] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:54.393283 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:54.349208 31208 raft_consensus.cc:3060] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:54.333807 30778 raft_consensus.cc:2804] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:47:54.349606 31213 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:54.388432 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:54.329169 31171 raft_consensus.cc:2749] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:54.324558 31003 consensus_queue.cc:1059] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260807 01:47:54.333560 31053 raft_consensus.cc:2749] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:47:54.428814 27214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57654 (request call id 1827) took 2259 ms. Trace:
I20260807 01:47:54.394749 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:54.430460 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:54.386961 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:54.431792 27103 raft_consensus.cc:2393] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 3: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
W20260807 01:47:54.432116 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 145472us
I20260807 01:47:54.432641 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:54.432860 27287 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:54.433748 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:54.396497 27067 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:54.399682 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:54.359854 27213 consensus_queue.cc:260] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:54.442584 27213 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:54.326454 30816 consensus_queue.cc:1059] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260807 01:47:54.360176 31187 raft_consensus.cc:2804] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:47:54.402318 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:54.360559 31111 raft_consensus.cc:2749] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:47:54.406001 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:54.365921 31150 raft_consensus.cc:2749] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 01:47:54.416492 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:54.417089 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:54.417717 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:54.382834 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:54.475739 27212 raft_consensus.cc:2393] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 5: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:54.333941 30941 consensus_queue.cc:1048] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260807 01:47:54.447677 27105 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1666) took 2133 ms. Trace:
I20260807 01:47:54.434545 27172 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:47:54.459270 27099 raft_consensus.cc:2393] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 5: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:47:54.429265 27214 rpcz_store.cc:276] 0807 01:47:52.175809 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.252294 (+2076485us) service_pool.cc:224] Handling call
0807 01:47:54.304004 (+ 51710us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.351340 (+ 47336us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:54.351370 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:54.351413 (+ 43us) raft_consensus.cc:1645] Marking committed up to 3
0807 01:47:54.351454 (+ 41us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.428549 (+ 77095us) spinlock_profiling.cc:243] Waited 47.1 ms on lock 0x7b54000c5dd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:47:54.428601 (+ 52us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:54.428752 (+ 151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108356480}
I20260807 01:47:54.353277 31207 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:54.463718 28709 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [term 1 LEARNER]: Advancing to term 3
I20260807 01:47:54.401499 30778 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:54.359406 31193 consensus_queue.cc:1059] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260807 01:47:54.467855 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:54.467877 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:54.353526 31194 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:54.348975 30942 consensus_queue.cc:1059] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260807 01:47:54.366102 30928 consensus_queue.cc:1048] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260807 01:47:54.464177 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:54.476229 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:54.476259 27105 rpcz_store.cc:276] 0807 01:47:52.314373 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.155238 (+1840865us) service_pool.cc:224] Handling call
0807 01:47:54.155526 (+ 288us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.155728 (+ 202us) raft_consensus.cc:1534] Early marking committed up to index 9
0807 01:47:54.332010 (+176282us) log.cc:844] Serialized 14 byte log entry
0807 01:47:54.447304 (+115294us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:54.447360 (+ 56us) raft_consensus.cc:1645] Marking committed up to 9
0807 01:47:54.447411 (+ 51us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.447475 (+ 64us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:54.447615 (+ 140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4980,"raft.queue_time_us":59551,"thread_start_us":252153,"threads_started":2}
I20260807 01:47:54.404961 31176 consensus_queue.cc:1059] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.090s
I20260807 01:47:54.482818 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:54.508234 28715 raft_consensus.cc:2393] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 3: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:47:54.412997 31213 raft_consensus.cc:3060] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:54.487885 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:54.390033 31024 consensus_queue.cc:1059] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
W20260807 01:47:54.490243 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 57467.3us
I20260807 01:47:54.395284 31203 consensus_queue.cc:1048] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.075s
I20260807 01:47:54.460263 30787 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:54.400574 31204 consensus_queue.cc:1048] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.083s
I20260807 01:47:54.403635 31090 consensus_queue.cc:1059] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.085s
I20260807 01:47:54.451330 31236 raft_consensus.cc:2804] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:47:54.508770 27213 raft_consensus.cc:1275] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260807 01:47:54.561875 27213 pending_rounds.cc:85] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f: Aborting all ops after (but not including) 4
I20260807 01:47:54.562198 27213 pending_rounds.cc:107] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260807 01:47:54.562467 27213 raft_consensus.cc:2889] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 01:47:54.563249 27213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57654 (request call id 1831) took 2274 ms. Trace:
I20260807 01:47:54.488349 31052 raft_consensus.cc:2804] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:47:54.568696 31052 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:54.569146 31052 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:54.512216 28709 raft_consensus.cc:1275] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260807 01:47:54.512768 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 158912us
I20260807 01:47:54.383405 31201 consensus_queue.cc:260] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:54.578789 30816 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:54.579440 30816 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:54.514134 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:54.585850 27230 raft_consensus.cc:2393] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 3: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:54.586966 31236 raft_consensus.cc:697] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 7 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:54.588129 31236 consensus_queue.cc:237] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:54.590498 31158 consensus_queue.cc:1059] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260807 01:47:54.384624 28707 rpcz_store.cc:276] 0807 01:47:53.227923 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:53.984265 (+756342us) service_pool.cc:224] Handling call
0807 01:47:53.984729 (+ 464us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.190683 (+205954us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.302291 (+111608us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b5400566dd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:47:54.306528 (+ 4237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":233276672}
I20260807 01:47:54.513337 27287 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:54.550765 27102 raft_consensus.cc:2393] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 6: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:47:54.495715 30778 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:54.476781 31187 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:54.483481 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:54.500018 31053 consensus_queue.cc:1059] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.114s
I20260807 01:47:54.558799 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:54.496407 31223 raft_consensus.cc:2749] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:47:54.449493 31115 consensus_queue.cc:1059] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260807 01:47:54.563649 27213 rpcz_store.cc:276] 0807 01:47:52.288912 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.269573 (+1980661us) service_pool.cc:224] Handling call
0807 01:47:54.317458 (+ 47885us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.562953 (+245495us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.563168 (+ 215us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 01:47:54.566047 27141 debug-util.cc:398] Leaking SignalData structure 0x7b080026fc80 after lost signal to thread 24600
I20260807 01:47:54.581944 31194 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:54.480823 31238 raft_consensus.cc:2804] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260807 01:47:54.732204 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 75908.3us
I20260807 01:47:54.566006 31026 consensus_queue.cc:1059] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.207s
I20260807 01:47:54.790642 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:54.790794 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:54.791538 27214 raft_consensus.cc:3060] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
W20260807 01:47:54.730379 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 252403us
I20260807 01:47:54.558290 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:54.792434 27216 raft_consensus.cc:3060] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:54.738822 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
W20260807 01:47:54.769955 27141 debug-util.cc:398] Leaking SignalData structure 0x7b080026f6c0 after lost signal to thread 27011
I20260807 01:47:54.755034 31187 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:47:54.769083 31158 consensus_queue.cc:1243] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer 3262a9d183914b699227fdba0ca5bed3 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260807 01:47:54.794454 27141 debug-util.cc:398] Leaking SignalData structure 0x7b080026fc00 after lost signal to thread 27146
I20260807 01:47:54.793344 27230 raft_consensus.cc:3060] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:54.562932 30787 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:54.802000 27141 debug-util.cc:398] Leaking SignalData structure 0x7b080026fce0 after lost signal to thread 27148
W20260807 01:47:54.803246 27141 debug-util.cc:398] Leaking SignalData structure 0x7b0800150e20 after lost signal to thread 27269
W20260807 01:47:54.804661 27141 debug-util.cc:398] Leaking SignalData structure 0x7b0800325da0 after lost signal to thread 27378
W20260807 01:47:54.804967 27141 debug-util.cc:398] Leaking SignalData structure 0x7b08002b27a0 after lost signal to thread 27379
W20260807 01:47:54.806694 27141 debug-util.cc:398] Leaking SignalData structure 0x7b08003a14c0 after lost signal to thread 28748
W20260807 01:47:54.807052 27141 debug-util.cc:398] Leaking SignalData structure 0x7b08002cd920 after lost signal to thread 28750
I20260807 01:47:54.552362 31192 consensus_queue.cc:237] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:54.772079 31238 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:54.822424 31238 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:54.758891 31203 consensus_queue.cc:1048] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260807 01:47:54.818193 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
W20260807 01:47:54.820719 30742 consensus_meta.cc:287] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 0.641s user 0.010s sys 0.000s
W20260807 01:47:54.825793 31175 consensus_meta.cc:287] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.553s user 0.009s sys 0.000s
I20260807 01:47:54.777805 30928 consensus_queue.cc:1059] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260807 01:47:54.563922 31231 consensus_queue.cc:237] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:54.816835 31007 consensus_meta.cc:287] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade: Time spent flushing consensus metadata: real 0.631s user 0.011s sys 0.000s
W20260807 01:47:54.733057 27229 consensus_meta.cc:287] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.555s user 0.013s sys 0.000s
I20260807 01:47:54.841457 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:54.854873 27229 raft_consensus.cc:2468] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:47:54.855347 27229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41230 (request call id 422) took 1173 ms. Trace:
I20260807 01:47:54.810906 31053 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:47:54.861470 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 93144.7us
I20260807 01:47:54.853111 31261 raft_consensus.cc:2804] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:47:54.811296 31026 consensus_queue.cc:1243] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer 3262a9d183914b699227fdba0ca5bed3 log is divergent from this leader: its last log entry 7.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260807 01:47:54.855660 27229 rpcz_store.cc:276] 0807 01:47:53.681894 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.176290 (+494396us) service_pool.cc:224] Handling call
0807 01:47:54.855298 (+679008us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:54.851121 31164 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:54.840108 31175 raft_consensus.cc:2749] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:47:54.879159 27110 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 548) took 1875 ms. Trace:
W20260807 01:47:54.875723 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 426746us
I20260807 01:47:54.859117 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:54.858893 31053 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:54.887367 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:54.874965 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:54.890089 27222 raft_consensus.cc:1240] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:54.893329 27216 raft_consensus.cc:2468] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:54.897265 31208 raft_consensus.cc:2749] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:54.900278 31053 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:54.852022 31135 raft_consensus.cc:3055] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 8 LEADER]: Stepping down as leader of term 8
I20260807 01:47:54.882644 27230 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:47:54.885532 30787 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:54.854067 31244 raft_consensus.cc:3055] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 9 LEADER]: Stepping down as leader of term 9
I20260807 01:47:54.905045 31244 raft_consensus.cc:740] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:54.905942 31244 consensus_queue.cc:260] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:54.904554 27214 raft_consensus.cc:2468] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
I20260807 01:47:54.903780 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:54.907228 31244 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Advancing to term 10
I20260807 01:47:54.895709 27229 raft_consensus.cc:2468] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:47:54.873603 31261 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:54.902493 28660 leader_election.cc:304] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
W20260807 01:47:54.915783 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 402493us
I20260807 01:47:54.918524 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:54.907806 31135 raft_consensus.cc:740] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:54.920492 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:54.921814 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:54.873437 31007 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:54.923059 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:54.928750 27105 raft_consensus.cc:1275] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260807 01:47:54.924168 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:54.925549 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:54.916956 27287 leader_election.cc:304] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:54.932281 27104 raft_consensus.cc:2393] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 3: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
W20260807 01:47:54.934448 30962 consensus_meta.cc:287] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.748s user 0.005s sys 0.006s
W20260807 01:47:54.932101 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 329677us
I20260807 01:47:54.936297 27336 raft_consensus.cc:1240] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 5. Current term is 6. Ops: []
I20260807 01:47:54.936394 31184 raft_consensus.cc:3055] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 01:47:54.907522 30928 consensus_queue.cc:1059] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260807 01:47:54.914140 31261 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:54.903885 31052 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:54.941226 27287 leader_election.cc:304] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:54.921415 30742 raft_consensus.cc:2804] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:47:54.942505 27212 raft_consensus.cc:3055] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 6 LEADER]: Stepping down as leader of term 6
I20260807 01:47:54.943775 27212 raft_consensus.cc:740] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:54.943742 30742 raft_consensus.cc:493] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:54.948141 31052 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:54.935328 30962 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:54.948304 30742 raft_consensus.cc:3060] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
W20260807 01:47:54.950666 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 154432us
I20260807 01:47:54.951889 27212 consensus_queue.cc:260] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:54.939193 31184 raft_consensus.cc:740] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
W20260807 01:47:54.948975 31180 consensus_meta.cc:287] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade: Time spent flushing consensus metadata: real 0.786s user 0.004s sys 0.007s
I20260807 01:47:54.925985 31238 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:54.953565 30711 consensus_meta.cc:287] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade: Time spent flushing consensus metadata: real 0.675s user 0.005s sys 0.007s
I20260807 01:47:54.977526 27172 leader_election.cc:304] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:54.967052 28660 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:54.980194 27172 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:54.978003 27109 raft_consensus.cc:2393] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 8: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:47:54.977386 27212 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:54.984571 27099 raft_consensus.cc:1240] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 5. Current term is 6. Ops: []
I20260807 01:47:54.984866 28660 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:54.945420 31255 raft_consensus.cc:2804] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:54.959134 31053 consensus_queue.cc:1059] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
W20260807 01:47:54.986667 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 45549.2us
I20260807 01:47:54.986348 31258 raft_consensus.cc:2749] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:54.987628 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:54.987674 27287 leader_election.cc:304] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:54.988313 27104 raft_consensus.cc:2376] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6. Re-sending same reply.
I20260807 01:47:54.990737 31258 raft_consensus.cc:2749] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:47:54.991221 31179 raft_consensus.cc:3055] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:54.991676 31179 raft_consensus.cc:740] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:54.992688 31256 raft_consensus.cc:2804] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:55.003859 30778 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:55.004565 31244 raft_consensus.cc:2749] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:55.005399 31244 raft_consensus.cc:2749] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 01:47:55.014016 28660 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:54.981412 31180 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:55.053597 27212 raft_consensus.cc:1275] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260807 01:47:54.983709 30711 raft_consensus.cc:515] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:55.082312 30816 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:54.987293 31255 raft_consensus.cc:697] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:55.212306 31256 raft_consensus.cc:697] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:55.266942 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
W20260807 01:47:55.272230 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:55.272527 27212 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Advancing to term 8
W20260807 01:47:55.243135 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 255492us
I20260807 01:47:54.959317 31135 consensus_queue.cc:260] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:55.073293 31184 consensus_queue.cc:260] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:54.976543 31203 consensus_queue.cc:1059] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
W20260807 01:47:55.015146 27063 leader_election.cc:336] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.529s (SENT)
I20260807 01:47:55.277832 27063 leader_election.cc:304] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
W20260807 01:47:55.278787 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 263663us
I20260807 01:47:54.992756 31179 consensus_queue.cc:260] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:55.250104 31236 raft_consensus.cc:2804] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:47:55.279677 31236 raft_consensus.cc:697] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:55.280004 31184 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:55.281955 27287 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:54.986539 31052 consensus_queue.cc:1059] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
W20260807 01:47:55.245043 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 264016us
W20260807 01:47:55.245293 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 231371us
I20260807 01:47:55.275683 27064 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:55.284801 27172 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:55.245846 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:55.286723 27216 raft_consensus.cc:2393] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 7: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:55.252264 30711 leader_election.cc:290] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:55.248817 30928 raft_consensus.cc:2749] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260807 01:47:55.284041 27212 raft_consensus.cc:2468] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 8.
I20260807 01:47:55.295938 31135 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:47:55.262074 30778 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:55.279124 31254 raft_consensus.cc:2749] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:47:55.250380 31007 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:55.297865 27172 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:55.300453 31150 raft_consensus.cc:2749] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:55.294739 31189 consensus_queue.cc:1048] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260807 01:47:55.299355 27064 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:55.247886 30888 consensus_queue.cc:1059] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260807 01:47:55.295652 31204 raft_consensus.cc:2804] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:47:55.308532 31259 consensus_queue.cc:1048] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260807 01:47:55.315246 30742 raft_consensus.cc:515] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:55.309299 30962 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:55.318064 31244 raft_consensus.cc:3055] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:47:55.306345 31194 consensus_queue.cc:1048] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260807 01:47:55.318663 31244 raft_consensus.cc:740] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:55.303021 30928 raft_consensus.cc:493] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:55.320348 30742 leader_election.cc:290] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:55.319844 31244 consensus_queue.cc:260] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:55.307277 31271 raft_consensus.cc:2804] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:47:55.320375 30928 raft_consensus.cc:515] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:55.331908 31260 raft_consensus.cc:2749] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:55.332180 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:55.315608 31223 raft_consensus.cc:2804] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 01:47:55.334139 30928 leader_election.cc:290] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:47:55.335000 31232 log.cc:927] Time spent T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 1.046s user 0.014s sys 0.021s
I20260807 01:47:55.334373 31223 raft_consensus.cc:697] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 8 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:55.336673 27101 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1651) took 3200 ms. Trace:
I20260807 01:47:55.337286 27101 rpcz_store.cc:276] 0807 01:47:52.136516 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.132724 (+1996208us) service_pool.cc:224] Handling call
0807 01:47:54.133103 (+ 379us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:47:54.133472 (+ 369us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:54.133500 (+ 28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:47:54.133954 (+ 454us) log.cc:844] Serialized 26 byte log entry
0807 01:47:54.289878 (+155924us) raft_consensus.cc:1645] Marking committed up to 3
0807 01:47:54.289916 (+ 38us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.289979 (+ 63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:47:55.312358 (+1022379us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b440002250c. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d5223e855 00007f7d5223f272 00007f7d54fc93bb 00007f7d54fc4c9f 00007f7d54fb8a5e 00007f7d54fb7da6 00007f7d54fb79c8 00007f7d54fc1411 00007f7d54fc13c9 00007f7d54fc1359
0807 01:47:55.336313 (+ 23955us) raft_consensus.cc:1685] finished
0807 01:47:55.336335 (+ 22us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:55.336609 (+ 274us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14579,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":112,"spinlock_wait_cycles":27297536,"thread_start_us":96740,"threads_started":1,"wal-append.queue_time_us":154267}
I20260807 01:47:55.310756 30711 consensus_queue.cc:1059] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260807 01:47:55.343393 28713 raft_consensus.cc:1240] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 8. Current term is 9. Ops: []
I20260807 01:47:54.880606 27110 rpcz_store.cc:276] 0807 01:47:53.003556 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.175521 (+1171965us) service_pool.cc:224] Handling call
0807 01:47:54.175811 (+ 290us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.176003 (+ 192us) raft_consensus.cc:1534] Early marking committed up to index 6
0807 01:47:54.419090 (+243087us) log.cc:844] Serialized 14 byte log entry
0807 01:47:54.878800 (+459710us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:54.878854 (+ 54us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:47:54.878904 (+ 50us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:54.878971 (+ 67us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:54.879101 (+ 130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":376962,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":1462,"thread_start_us":701757,"threads_started":2}
I20260807 01:47:55.252624 31187 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:55.347563 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:55.322825 31255 consensus_queue.cc:237] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:55.285712 31090 consensus_queue.cc:1059] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
W20260807 01:47:55.330139 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:55.331094 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:55.348228 28709 raft_consensus.cc:2468] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:47:55.355068 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:55.356307 31117 raft_consensus.cc:3055] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:55.359977 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:55.360678 27110 raft_consensus.cc:2468] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
W20260807 01:47:55.362533 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 79794.8us
I20260807 01:47:55.364344 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:55.319749 31238 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:55.367612 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:55.365114 27110 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:47:55.298379 31192 consensus_queue.cc:1059] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260807 01:47:55.367998 27175 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
W20260807 01:47:55.371821 27063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:55.361436 31117 raft_consensus.cc:740] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:55.372406 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:55.372761 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:55.374444 27172 leader_election.cc:304] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:55.377400 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:55.380976 27172 leader_election.cc:304] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:55.378393 27110 raft_consensus.cc:2468] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:47:55.384356 27172 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:55.362226 31204 raft_consensus.cc:697] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
W20260807 01:47:55.373962 31279 log.cc:927] Time spent T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df: Append to log took a long time: real 0.106s user 0.000s sys 0.002s
I20260807 01:47:55.373535 28713 raft_consensus.cc:1240] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5. Ops: []
I20260807 01:47:55.375308 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:55.385592 31052 raft_consensus.cc:2749] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260807 01:47:55.449755 27063 leader_election.cc:336] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.670s (SENT)
I20260807 01:47:55.486184 27063 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
W20260807 01:47:55.487541 27063 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.700s (SENT) [suppressed 30 similar messages]
I20260807 01:47:55.375398 31203 raft_consensus.cc:2804] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:47:55.492549 27172 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:55.366396 31256 consensus_queue.cc:237] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:55.390996 31024 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.637s user 0.016s sys 0.025s
I20260807 01:47:55.388952 31260 raft_consensus.cc:2749] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:55.392086 31150 raft_consensus.cc:2804] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:47:55.336710 31223 consensus_queue.cc:237] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:55.504351 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:55.505899 31271 raft_consensus.cc:697] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:55.395560 31117 consensus_queue.cc:260] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:55.499159 31213 consensus_meta.cc:287] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.935s user 0.010s sys 0.013s
W20260807 01:47:55.562748 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 196057us
W20260807 01:47:55.499229 31024 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.766s user 0.017s sys 0.025s
I20260807 01:47:55.561908 31187 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:55.495954 31090 raft_consensus.cc:2749] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260807 01:47:55.573252 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 80793.9us
W20260807 01:47:55.573697 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:55.574735 31231 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:47:55.571586 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 175982us
I20260807 01:47:55.497777 31203 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:55.503999 31150 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:55.577317 31150 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:47:55.580188 31203 raft_consensus.cc:3060] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:55.582064 27287 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:55.582196 28660 leader_election.cc:304] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:55.575587 31231 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:55.584195 31277 log.cc:927] Time spent T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.081s user 0.000s sys 0.000s
I20260807 01:47:55.589836 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:55.590662 27326 raft_consensus.cc:2468] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:55.593473 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:55.593967 27328 raft_consensus.cc:1240] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6. Ops: []
I20260807 01:47:55.596350 28709 raft_consensus.cc:1240] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6. Ops: []
I20260807 01:47:55.597461 27333 raft_consensus.cc:1240] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 3. Current term is 4. Ops: []
I20260807 01:47:55.598047 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:55.599131 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:55.600099 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:55.600302 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:55.601506 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:55.602350 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:55.602522 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:55.603257 27212 raft_consensus.cc:2468] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:47:55.608068 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:55.609067 27212 raft_consensus.cc:2410] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
W20260807 01:47:55.570539 31261 consensus_meta.cc:287] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade: Time spent flushing consensus metadata: real 0.621s user 0.010s sys 0.007s
I20260807 01:47:55.611491 27333 raft_consensus.cc:1240] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260807 01:47:55.609688 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:47:55.612334 27287 connection.cc:256] RPC call timeout handler was delayed by -2.0368240142706782s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16180000000000003s for a response
W20260807 01:47:55.612980 27287 connection.cc:256] RPC call timeout handler was delayed by -1.8199240141548216s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1859s for a response
I20260807 01:47:55.611395 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:47:55.613601 27287 connection.cc:256] RPC call timeout handler was delayed by -1.7290240142028779s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.196s for a response
W20260807 01:47:55.614094 27287 connection.cc:256] RPC call timeout handler was delayed by -1.6876240142155439s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2006s for a response
W20260807 01:47:55.614501 27287 connection.cc:256] RPC call timeout handler was delayed by -1.6654100781306624s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1543s for a response
W20260807 01:47:55.614857 27287 connection.cc:256] RPC call timeout handler was delayed by -1.6345240140799433s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20650000000000002s for a response
W20260807 01:47:55.615245 27287 connection.cc:256] RPC call timeout handler was delayed by -1.4926100780721754s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17350000000000002s for a response
W20260807 01:47:55.615612 27287 connection.cc:256] RPC call timeout handler was delayed by -1.4086240141186863s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2316s for a response
W20260807 01:47:55.615958 27287 connection.cc:256] RPC call timeout handler was delayed by -1.3630100782029331s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1879s for a response
W20260807 01:47:55.616318 27287 connection.cc:256] RPC call timeout handler was delayed by -1.2259403290227056s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
W20260807 01:47:55.616680 27287 connection.cc:256] RPC call timeout handler was delayed by -1.2043403291609138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15380000000000002s for a response
W20260807 01:47:55.617054 27287 connection.cc:256] RPC call timeout handler was delayed by -1.1494403290562332s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15990000000000001s for a response
W20260807 01:47:55.617499 27287 connection.cc:256] RPC call timeout handler was delayed by -1.1476403290871531s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
I20260807 01:47:55.613344 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:47:55.617949 27287 connection.cc:256] RPC call timeout handler was delayed by -1.1368403290398419s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1613s for a response
W20260807 01:47:55.618484 27287 connection.cc:256] RPC call timeout handler was delayed by -1.0738403291907161s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1683s for a response
I20260807 01:47:55.619732 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:55.620575 27325 raft_consensus.cc:2393] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 3: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:55.585800 31207 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:55.625633 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:55.622107 27100 raft_consensus.cc:2393] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 3: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:47:55.622138 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:55.626924 27331 raft_consensus.cc:1240] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 1. Current term is 2. Ops: []
I20260807 01:47:55.384753 31236 consensus_queue.cc:237] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:55.625216 27337 raft_consensus.cc:1240] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260807 01:47:55.570829 31260 raft_consensus.cc:2749] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:47:55.630069 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:55.631151 27105 raft_consensus.cc:2393] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 6: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:47:55.626430 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:55.612527 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:55.631664 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:55.642204 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:55.636585 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:55.643131 27326 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:47:55.635821 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:55.634621 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:55.647204 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:55.634714 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:55.595993 31180 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:55.652560 27105 raft_consensus.cc:2393] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 10: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
W20260807 01:47:55.624171 31266 log.cc:927] Time spent T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.350s user 0.001s sys 0.001s
I20260807 01:47:55.572980 31213 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:55.646875 30778 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:55.657740 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:55.621371 31192 raft_consensus.cc:2804] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:47:55.656931 31289 consensus_queue.cc:1059] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260807 01:47:55.671689 31289 raft_consensus.cc:3055] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:55.672215 31289 raft_consensus.cc:740] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:55.673924 31289 consensus_queue.cc:260] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:55.676828 31289 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:55.687500 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:55.644301 31194 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:55.647850 31231 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:47:55.693478 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 53650.6us
I20260807 01:47:55.624765 31187 consensus_queue.cc:1048] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260807 01:47:55.629370 31204 consensus_queue.cc:237] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:55.667264 31192 raft_consensus.cc:493] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:55.624564 31261 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:55.719765 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:55.669529 30778 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:55.708261 31122 raft_consensus.cc:493] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:55.712672 31192 raft_consensus.cc:3060] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:55.696600 31194 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:55.727224 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:55.731179 27327 raft_consensus.cc:1240] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260807 01:47:55.731869 27326 raft_consensus.cc:2393] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 10: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:55.697738 31090 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:55.733783 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:55.735404 27103 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 7.
I20260807 01:47:55.708424 31271 consensus_queue.cc:237] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:55.761508 31272 raft_consensus.cc:2804] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:55.762707 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:55.763571 31272 raft_consensus.cc:697] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:55.763720 27325 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:47:55.743196 31184 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:55.731130 31122 raft_consensus.cc:515] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:55.846055 31102 raft_consensus.cc:3055] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:55.959111 27108 raft_consensus.cc:1240] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6. Ops: []
W20260807 01:47:55.974368 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:47:55.952101 27067 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.470s user 0.006s sys 0.043s
W20260807 01:47:56.231820 31103 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.838s user 0.032s sys 0.068s
I20260807 01:47:55.995316 27226 raft_consensus.cc:3055] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:47:56.379868 31102 raft_consensus.cc:740] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
W20260807 01:47:55.958918 27102 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 1650) took 3822 ms (client timeout 2873 ms). Trace:
W20260807 01:47:55.827310 31135 consensus_meta.cc:287] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade: Time spent flushing consensus metadata: real 0.531s user 0.013s sys 0.000s
I20260807 01:47:56.438045 27226 raft_consensus.cc:740] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:55.979482 27218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57662 (request call id 2590) took 1600 ms. Trace:
I20260807 01:47:56.349654 31254 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:55.817493 31171 raft_consensus.cc:3055] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:47:56.221691 31187 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
W20260807 01:47:55.860296 31239 log.cc:927] Time spent T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 1.298s user 0.015s sys 0.045s
W20260807 01:47:55.844060 31153 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
W20260807 01:47:56.443058 31103 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.054s user 0.032s sys 0.077s
I20260807 01:47:55.848603 31281 raft_consensus.cc:3055] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Stepping down as leader of term 2
W20260807 01:47:56.380417 31203 consensus_meta.cc:287] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.680s user 0.014s sys 0.002s
I20260807 01:47:55.981614 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:56.170785 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:56.201169 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:56.379622 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:55.846450 31176 raft_consensus.cc:3055] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:56.458251 31158 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:56.458539 31171 raft_consensus.cc:740] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:55.777554 31150 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:56.459134 31294 log.cc:927] Time spent T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.099s user 0.001s sys 0.001s
I20260807 01:47:56.459259 28710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48362 (request call id 745) took 2133 ms. Trace:
I20260807 01:47:55.857592 31192 raft_consensus.cc:515] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:56.460696 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:55.840094 31269 raft_consensus.cc:3055] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Stepping down as leader of term 3
W20260807 01:47:56.441437 27102 rpcz_store.cc:269] 0807 01:47:52.136273 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:53.765539 (+1629266us) service_pool.cc:224] Handling call
0807 01:47:55.958046 (+2192507us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b54003174d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:47:55.958549 (+ 503us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":967301504}
W20260807 01:47:56.461901 28660 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.739s user 0.013s sys 0.007s
W20260807 01:47:56.462339 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:47:56.462898 31255 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.725s user 0.009s sys 0.012s
I20260807 01:47:56.462568 31254 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:56.463793 27338 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.806s user 0.015s sys 0.015s
I20260807 01:47:55.773845 30778 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:55.793099 31213 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:56.469246 31102 consensus_queue.cc:260] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:56.471118 31117 raft_consensus.cc:3060] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:55.860625 31243 raft_consensus.cc:3055] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:55.775547 31194 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:56.484462 27230 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:47:56.471524 27226 consensus_queue.cc:260] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:56.494756 28660 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.772s user 0.013s sys 0.007s
I20260807 01:47:55.836282 31244 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
W20260807 01:47:56.500262 31255 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.766s user 0.009s sys 0.012s
I20260807 01:47:56.489493 28710 rpcz_store.cc:276] 0807 01:47:54.325952 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.326102 (+ 150us) service_pool.cc:224] Handling call
0807 01:47:54.326404 (+ 302us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.473932 (+147528us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:54.473968 (+ 36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:54.474013 (+ 45us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:47:54.474053 (+ 40us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:56.458507 (+1984454us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54006077d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:47:56.458580 (+ 73us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:56.458959 (+ 379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":338773760}
W20260807 01:47:56.500082 27338 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.843s user 0.015s sys 0.015s
I20260807 01:47:56.507161 27216 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
W20260807 01:47:56.503120 27105 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.688s user 0.007s sys 0.013s
W20260807 01:47:56.505643 27063 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.727s user 0.019s sys 0.029s
W20260807 01:47:56.506619 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 784038us
I20260807 01:47:56.512928 31269 raft_consensus.cc:740] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
W20260807 01:47:56.512516 27174 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:47:56.517632 27063 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.840s user 0.020s sys 0.030s
W20260807 01:47:56.497766 31223 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s user 0.000s sys 0.014s
W20260807 01:47:56.523029 27105 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.809s user 0.008s sys 0.013s
I20260807 01:47:56.528364 27331 raft_consensus.cc:2393] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 3: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
W20260807 01:47:56.507014 31236 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.749s user 0.009s sys 0.029s
W20260807 01:47:56.512765 30816 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.726s user 0.006s sys 0.030s
I20260807 01:47:56.530911 31179 raft_consensus.cc:3060] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Advancing to term 3
W20260807 01:47:56.533221 27067 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.051s user 0.017s sys 0.049s
I20260807 01:47:56.530431 31171 consensus_queue.cc:260] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:56.531388 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:56.531795 31243 raft_consensus.cc:740] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:56.529650 31158 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:56.530808 27108 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 2917) took 3488 ms. Trace:
W20260807 01:47:56.516211 31279 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.754s user 0.007s sys 0.006s
I20260807 01:47:56.443615 27218 rpcz_store.cc:276] 0807 01:47:54.373520 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.373680 (+ 160us) service_pool.cc:224] Handling call
0807 01:47:54.374135 (+ 455us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:47:55.268275 (+894140us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:55.268306 (+ 31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:47:55.276331 (+ 8025us) log.cc:844] Serialized 26 byte log entry
0807 01:47:55.296420 (+ 20089us) raft_consensus.cc:1645] Marking committed up to 5
0807 01:47:55.296460 (+ 40us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:55.826794 (+530334us) spinlock_profiling.cc:243] Waited 4.02 ms on lock 0x7b180031ca20. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d5b233f31 00007f7d5b2406d1 00007f7d52317d33 00007f7d5231a121 00007f7d5231a0d9 00007f7d5231a069 00007f7d5231a031 00007f7d5231932d 00000000003aa554 00000000003aa399 00007f7d522fb1ea
0807 01:47:55.973231 (+146437us) spinlock_profiling.cc:243] Waited 893 ms on lock 0x7b54000f23d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:47:55.973340 (+ 109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:47:55.973499 (+ 159us) raft_consensus.cc:1685] finished
0807 01:47:55.973544 (+ 45us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:55.974028 (+ 484us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4442,"raft.queue_time_us":5194,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":12892,"spinlock_wait_cycles":2063776896,"thread_start_us":18225,"threads_started":1,"wal-append.queue_time_us":22787}
W20260807 01:47:56.528196 31223 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s user 0.000s sys 0.015s
I20260807 01:47:56.567718 27330 raft_consensus.cc:2393] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 6: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:56.572501 27337 raft_consensus.cc:2468] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
W20260807 01:47:56.562628 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 105780us
I20260807 01:47:56.527878 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:56.567173 27108 rpcz_store.cc:276] 0807 01:47:53.041870 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.334953 (+1293083us) service_pool.cc:224] Handling call
0807 01:47:55.958484 (+1623531us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:56.437971 (+479487us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:56.530498 (+ 92527us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54003174d0. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541dc8d 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:47:56.530680 (+ 182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3732501760}
W20260807 01:47:56.573827 31279 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.936s user 0.007s sys 0.008s
I20260807 01:47:56.562260 31281 raft_consensus.cc:740] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
W20260807 01:47:56.544919 31236 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.788s user 0.010s sys 0.030s
W20260807 01:47:56.566920 30816 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.798s user 0.007s sys 0.030s
I20260807 01:47:56.562568 31254 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:47:56.564111 31270 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.691s user 0.025s sys 0.042s
I20260807 01:47:55.743599 31090 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:55.770820 31184 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:56.573078 31176 raft_consensus.cc:740] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:56.598217 27226 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:56.583704 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
W20260807 01:47:56.578656 31256 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.786s user 0.027s sys 0.060s
I20260807 01:47:56.538290 31187 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:56.624186 31270 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.859s user 0.025s sys 0.043s
I20260807 01:47:55.764991 31272 consensus_queue.cc:237] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
W20260807 01:47:56.625855 31256 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.926s user 0.028s sys 0.068s
I20260807 01:47:56.579892 31153 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:56.597180 31203 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:56.646055 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:56.646306 31269 consensus_queue.cc:260] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:56.648942 31192 leader_election.cc:290] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:56.650322 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:56.667659 27213 raft_consensus.cc:3060] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:56.651762 27099 raft_consensus.cc:1240] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5. Ops: []
I20260807 01:47:56.652710 31187 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:56.655052 31207 raft_consensus.cc:493] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:56.655800 31171 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:56.666863 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
W20260807 01:47:56.676144 28660 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:56.626180 31255 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:56.676743 28702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48350 (request call id 800) took 1270 ms. Trace:
I20260807 01:47:56.677021 31207 raft_consensus.cc:515] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:56.682735 31261 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:56.683763 30942 raft_consensus.cc:3055] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Stepping down as leader of term 2
W20260807 01:47:56.685004 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 2.28839e+06us
I20260807 01:47:56.687402 27102 raft_consensus.cc:1240] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 5. Current term is 6. Ops: []
I20260807 01:47:56.700232 27218 raft_consensus.cc:2376] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3. Re-sending same reply.
I20260807 01:47:56.688421 27325 raft_consensus.cc:1240] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:56.689222 31158 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:56.690330 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:56.687426 27103 raft_consensus.cc:1275] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260807 01:47:56.695482 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
W20260807 01:47:56.699208 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:56.709324 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:56.709143 31176 consensus_queue.cc:260] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:56.710494 31153 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:56.655288 31243 consensus_queue.cc:260] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:56.728948 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:56.729668 31243 raft_consensus.cc:3060] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:56.731864 27099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1679) took 1133 ms. Trace:
I20260807 01:47:56.732635 27099 rpcz_store.cc:276] 0807 01:47:55.598633 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:55.598789 (+ 156us) service_pool.cc:224] Handling call
0807 01:47:55.634229 (+ 35440us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:56.699976 (+1065747us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:56.731344 (+ 31368us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54003363d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:47:56.731771 (+ 427us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2338652416}
I20260807 01:47:56.734266 31003 raft_consensus.cc:3055] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:56.734992 31003 raft_consensus.cc:740] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:56.741245 31201 raft_consensus.cc:3060] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 1 FOLLOWER]: Advancing to term 2
I20260807 01:47:56.743671 31184 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:56.746949 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:56.750420 27327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59490 (request call id 2505) took 1154 ms. Trace:
I20260807 01:47:56.750896 27327 rpcz_store.cc:276] 0807 01:47:55.595572 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:55.595866 (+ 294us) service_pool.cc:224] Handling call
0807 01:47:55.634297 (+ 38431us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:47:55.737913 (+103616us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:56.749763 (+1011850us) spinlock_profiling.cc:243] Waited 94.6 ms on lock 0x7b540001dbd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:47:56.750196 (+ 433us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":217663872}
I20260807 01:47:56.752031 31122 leader_election.cc:290] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:56.753271 31281 consensus_queue.cc:260] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:56.734347 27100 raft_consensus.cc:1240] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 8 LEADER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 7. Current term is 8. Ops: []
I20260807 01:47:56.729029 27336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36604 (request call id 635) took 2528 ms. Trace:
I20260807 01:47:56.756009 31203 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:47:56.756662 31315 log.cc:927] Time spent T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.056s user 0.001s sys 0.000s
I20260807 01:47:56.757166 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:56.757576 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:56.758193 31090 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:56.754667 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:56.701845 30942 raft_consensus.cc:740] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:56.749729 31207 leader_election.cc:290] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:47:56.749719 28707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48362 (request call id 750) took 1402 ms (client timeout 1555 ms). Trace:
I20260807 01:47:56.768915 30942 consensus_queue.cc:260] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:56.771677 30942 raft_consensus.cc:3060] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:56.768477 27331 raft_consensus.cc:2393] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 2: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:56.765022 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:56.774729 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:47:56.779112 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 60429us
I20260807 01:47:56.780205 27213 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:56.734875 31150 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:56.782393 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:56.782956 31176 raft_consensus.cc:3060] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:56.787684 27327 raft_consensus.cc:1240] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 5. Current term is 6. Ops: []
W20260807 01:47:56.788690 27170 connection.cc:256] RPC call timeout handler was delayed by -1.2409806761424989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19830000000000003s for a response
I20260807 01:47:56.697078 28702 rpcz_store.cc:276] 0807 01:47:55.347444 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:55.356835 (+ 9391us) service_pool.cc:224] Handling call
0807 01:47:55.379199 (+ 22364us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:47:55.561665 (+182466us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:55.562058 (+ 393us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:47:55.569319 (+ 7261us) log.cc:844] Serialized 26 byte log entry
0807 01:47:55.710262 (+140943us) raft_consensus.cc:1645] Marking committed up to 2
0807 01:47:55.710335 (+ 73us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:55.710422 (+ 87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:47:56.535937 (+825515us) raft_consensus.cc:1685] finished
0807 01:47:56.544712 (+ 8775us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:56.617987 (+ 73275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17096,"raft.queue_time_us":106862,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":15993,"thread_start_us":126631,"threads_started":1,"wal-append.queue_time_us":660734}
I20260807 01:47:56.755714 27336 rpcz_store.cc:276] 0807 01:47:54.200488 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.215901 (+ 15413us) service_pool.cc:224] Handling call
0807 01:47:54.216266 (+ 365us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:54.936850 (+720584us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:56.728445 (+1791595us) spinlock_profiling.cc:243] Waited 720 ms on lock 0x7b54003002d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:47:56.728795 (+ 350us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1655454080}
I20260807 01:47:56.789021 31207 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
W20260807 01:47:56.768565 28707 rpcz_store.cc:269] 0807 01:47:55.346916 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:55.357016 (+ 10100us) service_pool.cc:224] Handling call
0807 01:47:56.689794 (+1332778us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b5800055598. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e0521 00007f7d522e03d2 00007f7d5b24c219 00007f7d5a39f640 00007f7d5a467ebb 00007f7d5a467f41 00007f7d5a415eb0 00007f7d5a415794 00007f7d5395e834 00007f7d5395e7c2
0807 01:47:56.749606 (+ 59812us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6203264}
I20260807 01:47:56.792908 31207 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:56.794369 27330 raft_consensus.cc:2393] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 6: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:56.794639 31024 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:56.796271 31024 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:56.803031 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:47:56.752068 31303 log.cc:927] Time spent T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.063s user 0.002s sys 0.000s
I20260807 01:47:56.800429 27337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33976 (request call id 2505) took 1081 ms. Trace:
W20260807 01:47:56.758265 31308 log.cc:927] Time spent T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.087s user 0.002s sys 0.002s
I20260807 01:47:56.729027 31269 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:56.808905 27337 rpcz_store.cc:276] 0807 01:47:55.719409 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:55.719539 (+ 130us) service_pool.cc:224] Handling call
0807 01:47:56.800120 (+1080581us) spinlock_profiling.cc:243] Waited 448 ms on lock 0x7b5400274554. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:47:56.800320 (+ 200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1031142784}
I20260807 01:47:56.800429 27338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33976 (request call id 2500) took 1164 ms. Trace:
I20260807 01:47:56.812832 27226 raft_consensus.cc:1275] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260807 01:47:56.754758 31281 raft_consensus.cc:3060] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
W20260807 01:47:56.817396 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:47:56.802443 31311 log.cc:927] Time spent T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.142s user 0.000s sys 0.001s
I20260807 01:47:56.821539 31291 raft_consensus.cc:493] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:56.826136 27216 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:47:56.822443 31291 raft_consensus.cc:515] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:56.828385 31299 log.cc:927] Time spent T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.150s user 0.000s sys 0.001s
I20260807 01:47:56.830509 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:47:56.830844 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:56.831385 27103 raft_consensus.cc:3060] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:56.828095 27287 leader_election.cc:304] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:56.834507 31235 raft_consensus.cc:2749] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:56.834741 31291 leader_election.cc:290] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:56.838199 31150 raft_consensus.cc:493] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
W20260807 01:47:56.835088 31193 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260807 01:47:56.846840 27226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57662 (request call id 2594) took 2452 ms. Trace:
I20260807 01:47:56.847788 27212 raft_consensus.cc:3060] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:56.736941 31003 consensus_queue.cc:260] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:56.847509 27226 rpcz_store.cc:276] 0807 01:47:54.374804 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:54.376537 (+ 1733us) service_pool.cc:224] Handling call
0807 01:47:54.376816 (+ 279us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:55.994991 (+1618175us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b54000e6b20. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d553961a0 00007f7d554205a0 00007f7d55421611 00007f7d5541e16f 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c
0807 01:47:56.814434 (+819443us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:56.827317 (+ 12883us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":437455232}
I20260807 01:47:56.855690 27218 raft_consensus.cc:2393] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 5: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:56.838924 31150 raft_consensus.cc:515] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:56.725708 31255 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:56.862561 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:56.864138 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:47:56.864434 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 53116.2us
I20260807 01:47:56.838950 31024 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:56.858812 31150 leader_election.cc:290] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:56.813423 27338 rpcz_store.cc:276] 0807 01:47:55.635778 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:55.635985 (+ 207us) service_pool.cc:224] Handling call
0807 01:47:55.647527 (+ 11542us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:55.647760 (+ 233us) raft_consensus.cc:1534] Early marking committed up to index 4
0807 01:47:56.580139 (+932379us) log.cc:844] Serialized 14 byte log entry
0807 01:47:56.727521 (+147382us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:47:56.766458 (+ 38937us) raft_consensus.cc:1645] Marking committed up to 4
0807 01:47:56.800052 (+ 33594us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:56.800187 (+ 135us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:56.800344 (+ 157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":923808,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":21195,"raft.queue_time_us":4162,"spinlock_wait_cycles":25178496,"thread_start_us":912324,"threads_started":2,"wal-append.queue_time_us":138511}
I20260807 01:47:56.853235 31291 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:56.873025 31024 consensus_queue.cc:1059] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260807 01:47:56.878619 28710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:34614
I20260807 01:47:56.879562 28710 raft_consensus.cc:606] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:56.879926 27102 raft_consensus.cc:2468] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
W20260807 01:47:56.879024 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:56.884389 27105 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1686) took 1275 ms. Trace:
I20260807 01:47:56.870119 31158 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:56.852843 31103 consensus_queue.cc:1059] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260807 01:47:56.894591 31158 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:56.879956 28710 raft_consensus.cc:614] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 01:47:56.888345 31150 raft_consensus.cc:3055] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:47:56.889441 31329 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:56.889307 31291 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:56.884930 31003 raft_consensus.cc:3060] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:56.885332 27105 rpcz_store.cc:276] 0807 01:47:55.608713 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:55.653268 (+ 44555us) service_pool.cc:224] Handling call
0807 01:47:55.653627 (+ 359us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:47:55.666336 (+ 12709us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:55.666370 (+ 34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:47:55.693206 (+ 26836us) log.cc:844] Serialized 26 byte log entry
0807 01:47:56.627481 (+934275us) raft_consensus.cc:1645] Marking committed up to 4
0807 01:47:56.638776 (+ 11295us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:56.658478 (+ 19702us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:47:56.868585 (+210107us) raft_consensus.cc:1685] finished
0807 01:47:56.868607 (+ 22us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:56.884318 (+ 15711us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4687,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":18133,"thread_start_us":819962,"threads_started":1,"wal-append.queue_time_us":787994}
I20260807 01:47:56.897053 27230 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 8.
I20260807 01:47:56.908613 27103 raft_consensus.cc:2468] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
I20260807 01:47:56.895285 30681 raft_consensus.cc:3055] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:47:56.914688 30681 raft_consensus.cc:740] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:56.916023 30681 consensus_queue.cc:260] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:56.902793 31150 raft_consensus.cc:740] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:56.918306 27170 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:56.920373 27100 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1688) took 1311 ms. Trace:
I20260807 01:47:56.920814 27100 rpcz_store.cc:276] 0807 01:47:55.609252 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:55.667006 (+ 57754us) service_pool.cc:224] Handling call
0807 01:47:55.667398 (+ 392us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:56.755419 (+1088021us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:56.920065 (+164646us) spinlock_profiling.cc:243] Waited 988 ms on lock 0x7b54003d6dd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:47:56.920295 (+ 230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2273007488}
W20260807 01:47:56.921888 31317 log.cc:927] Time spent T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.239s user 0.001s sys 0.000s
I20260807 01:47:56.922889 31053 raft_consensus.cc:3055] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:56.923296 31053 raft_consensus.cc:740] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:56.924329 27067 leader_election.cc:304] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
W20260807 01:47:56.925295 31304 log.cc:927] Time spent T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.237s user 0.001s sys 0.001s
W20260807 01:47:56.926618 24592 auto_leader_rebalancer.cc:467] leader transfer for tablet 51cef821f0284d8e9369e98d38f80e73 (from 57bb4985157447f0a3ab680fa0026f12 to 3262a9d183914b699227fdba0ca5bed3) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
W20260807 01:47:56.928124 28661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:56.928524 31207 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:56.929225 31207 raft_consensus.cc:2804] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:47:56.929647 31207 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:56.929582 31255 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:56.930102 31207 raft_consensus.cc:3060] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:56.935173 31153 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:47:56.918334 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:56.937191 27226 raft_consensus.cc:1240] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 4. Current term is 5. Ops: []
I20260807 01:47:56.924360 31053 consensus_queue.cc:260] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:56.939128 31053 raft_consensus.cc:3060] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:56.939383 31329 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:56.939944 31158 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:56.940884 31150 consensus_queue.cc:260] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:56.942494 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:56.943444 31090 raft_consensus.cc:2749] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:56.945393 31204 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:47:56.946692 31204 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:56.947852 27108 raft_consensus.cc:1240] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6. Ops: []
I20260807 01:47:56.949764 27102 raft_consensus.cc:1240] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260807 01:47:56.920832 27287 leader_election.cc:304] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:56.951422 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:47:56.960737 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:47:56.996999 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 76319.9us
W20260807 01:47:56.998476 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:56.999430 31184 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:57.000176 31184 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:57.003018 31184 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:57.004027 31289 raft_consensus.cc:2749] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:56.939992 31291 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:57.009056 31329 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:57.010524 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:57.011638 28709 raft_consensus.cc:2393] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 3: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:47:57.012506 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:57.013912 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:57.014724 28702 raft_consensus.cc:3060] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:57.014973 28708 raft_consensus.cc:1240] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 3. Current term is 4. Ops: []
I20260807 01:47:57.015678 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:57.018296 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:57.018981 27102 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:47:57.019163 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:57.019941 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:57.021988 27104 raft_consensus.cc:1240] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260807 01:47:57.021798 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:57.024855 26950 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46134 (request call id 415) took 6446 ms. Trace:
I20260807 01:47:57.025362 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:57.027151 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:57.019879 27100 raft_consensus.cc:3060] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:57.029006 26952 catalog_manager.cc:5697] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:57.031409 31102 raft_consensus.cc:3060] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:57.032228 28710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:34614
W20260807 01:47:57.033628 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 214966us
W20260807 01:47:57.036536 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:47:57.035408 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 148980us
I20260807 01:47:57.038339 27170 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
W20260807 01:47:57.039618 27063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:56.918025 30681 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:57.042903 28710 raft_consensus.cc:606] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:47:57.043527 28710 raft_consensus.cc:614] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260807 01:47:57.042538 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 217401us
W20260807 01:47:57.069635 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 71589.5us
I20260807 01:47:57.041090 31179 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
W20260807 01:47:57.072918 27174 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:57.073988 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:57.074782 28715 raft_consensus.cc:1240] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 3. Current term is 4. Ops: []
I20260807 01:47:57.082348 27110 raft_consensus.cc:2393] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 5: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
W20260807 01:47:57.084249 24592 auto_leader_rebalancer.cc:467] leader transfer for tablet 965af9378c6a4617b1d50f229ffc2c5b (from 57bb4985157447f0a3ab680fa0026f12 to fcf4e45defb94276bcfa68392711dade) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
W20260807 01:47:57.086885 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 247947us
I20260807 01:47:57.084993 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 01:47:57.085489 31329 raft_consensus.cc:2749] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 01:47:57.115521 28715 raft_consensus.cc:1240] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5. Ops: []
W20260807 01:47:57.117314 27174 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:47:57.122596 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 84424.2us
I20260807 01:47:57.123487 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:47:57.126387 31150 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:57.129109 27287 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:57.126579 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:57.130872 27102 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8.
W20260807 01:47:57.136214 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 164584us
I20260807 01:47:57.136713 31298 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:57.137442 31298 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.133993 31203 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:57.138504 31203 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.139676 27214 raft_consensus.cc:2468] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 2.
I20260807 01:47:57.140698 31213 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:57.141358 31213 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:57.141955 31190 consensus_queue.cc:1059] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:47:57.143559 31171 raft_consensus.cc:3055] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:47:57.144006 31171 raft_consensus.cc:740] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:57.145023 31171 consensus_queue.cc:260] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.145641 28660 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3
W20260807 01:47:57.146234 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:57.149652 27287 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
W20260807 01:47:57.150496 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:57.134733 28710 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 9.
I20260807 01:47:57.033111 26950 rpcz_store.cc:276] 0807 01:47:50.578616 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:50.579071 (+ 455us) service_pool.cc:224] Handling call
0807 01:47:57.024524 (+6445453us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 01:47:54.879705 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 01:47:54.912797 (+ 33092us) write_op.cc:432] Acquiring schema lock in shared mode
0807 01:47:54.912838 (+ 41us) write_op.cc:435] Acquired schema lock
0807 01:47:54.912855 (+ 17us) tablet.cc:662] Decoding operations
0807 01:47:54.921936 (+ 9081us) write_op.cc:620] Acquiring the partition lock for write op
0807 01:47:54.926411 (+ 4475us) write_op.cc:641] Partition lock acquired for write op
0807 01:47:54.926447 (+ 36us) tablet.cc:685] Acquiring locks for 3 operations
0807 01:47:54.926805 (+ 358us) tablet.cc:701] Row locks acquired
0807 01:47:54.926820 (+ 15us) write_op.cc:260] PREPARE: finished
0807 01:47:54.926982 (+ 162us) write_op.cc:270] Start()
0807 01:47:54.927102 (+ 120us) write_op.cc:276] Timestamp: P: 1786067274926926 usec, L: 0
0807 01:47:54.927125 (+ 23us) op_driver.cc:348] REPLICATION: starting
0807 01:47:54.977858 (+ 50733us) log.cc:844] Serialized 1345 byte log entry
0807 01:47:56.750223 (+1772365us) op_driver.cc:464] REPLICATION: finished
0807 01:47:56.992540 (+242317us) write_op.cc:301] APPLY: starting
0807 01:47:56.992669 (+ 129us) tablet.cc:1367] starting BulkCheckPresence
0807 01:47:56.992893 (+ 224us) tablet.cc:1370] finished BulkCheckPresence
0807 01:47:56.993053 (+ 160us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 01:47:56.993856 (+ 803us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 01:47:56.994293 (+ 437us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0807 01:47:56.994360 (+ 67us) write_op.cc:312] APPLY: finished
0807 01:47:56.994747 (+ 387us) log.cc:844] Serialized 35 byte log entry
0807 01:47:56.994980 (+ 233us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:47:56.995122 (+ 142us) write_op.cc:454] Released schema lock
0807 01:47:56.995196 (+ 74us) write_op.cc:341] FINISH: Updating metrics
0807 01:47:56.995757 (+ 561us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:47:56.995793 (+ 36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":440216,"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":205825,"apply.run_cpu_time_us":3207,"apply.run_wall_time_us":3207,"num_ops":3,"prepare.queue_time_us":510377,"prepare.run_cpu_time_us":21000,"prepare.run_wall_time_us":395572,"raft.queue_time_us":983170,"raft.run_cpu_time_us":33431,"raft.run_wall_time_us":233425,"replication_time_us":1741549,"spinlock_wait_cycles":7040,"thread_start_us":1758613,"threads_started":4,"wal-append.queue_time_us":285914}]]}
I20260807 01:47:57.161949 27064 leader_election.cc:304] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:57.167925 28706 raft_consensus.cc:1240] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 7 LEADER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 6. Current term is 7. Ops: []
I20260807 01:47:57.162564 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:57.172016 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:57.151569 31204 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:57.116672 31179 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.176522 31223 raft_consensus.cc:2804] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:47:57.177017 31272 raft_consensus.cc:2749] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:57.177551 31290 raft_consensus.cc:2749] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:47:57.178260 31254 consensus_queue.cc:1048] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260807 01:47:57.178992 31255 consensus_queue.cc:1059] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260807 01:47:57.192960 31255 raft_consensus.cc:3055] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:57.193420 31255 raft_consensus.cc:740] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:57.193584 27101 raft_consensus.cc:2468] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:47:57.180035 27337 raft_consensus.cc:1275] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260807 01:47:57.184290 27289 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:57.136423 31306 consensus_queue.cc:1059] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260807 01:47:57.195907 30888 raft_consensus.cc:2804] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 01:47:57.196341 31306 raft_consensus.cc:3055] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:47:57.196893 31306 raft_consensus.cc:740] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:57.196893 31223 raft_consensus.cc:697] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:57.179279 27064 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
W20260807 01:47:57.208796 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:47:57.201895 31244 consensus_meta.cc:287] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.560s user 0.017s sys 0.000s
I20260807 01:47:57.198128 31223 consensus_queue.cc:237] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:57.196398 30888 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:57.217262 27100 raft_consensus.cc:2468] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:57.219683 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:47:57.223954 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 44822.2us
I20260807 01:47:57.228417 31102 raft_consensus.cc:2804] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 01:47:57.237525 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:57.237818 31102 raft_consensus.cc:697] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 8 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:57.238601 28715 raft_consensus.cc:2393] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 3: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:47:57.238911 31213 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:57.229121 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:57.239876 30854 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:57.240561 30854 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:57.242395 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:47:57.242981 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:47:57.243338 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:57.243862 31329 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:57.174530 31243 consensus_queue.cc:1059] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260807 01:47:57.244462 31329 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:57.256769 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 107234us
I20260807 01:47:57.260042 28660 leader_election.cc:304] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3
W20260807 01:47:57.269378 27063 leader_election.cc:336] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.555s (SENT)
I20260807 01:47:57.270289 27063 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:57.230454 30888 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Advancing to term 10
I20260807 01:47:57.231954 31171 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
W20260807 01:47:57.282982 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 113399us
W20260807 01:47:57.282966 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:57.197954 31306 consensus_queue.cc:260] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.288250 27212 raft_consensus.cc:2468] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:47:57.292506 31254 raft_consensus.cc:2749] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:47:57.295881 27212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41230 (request call id 438) took 1024 ms. Trace:
I20260807 01:47:57.296495 27212 rpcz_store.cc:276] 0807 01:47:56.274916 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:56.694944 (+420028us) service_pool.cc:224] Handling call
0807 01:47:57.291874 (+596930us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:47:57.299026 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:57.301798 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:57.304303 27230 raft_consensus.cc:2468] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:47:57.306061 27331 raft_consensus.cc:1240] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:57.309930 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:57.311210 27287 leader_election.cc:304] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:57.313860 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:57.313537 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:57.294399 31322 raft_consensus.cc:493] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:57.238936 31102 consensus_queue.cc:237] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:57.324196 31161 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:47:57.326977 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 111733us
W20260807 01:47:57.328209 31176 consensus_meta.cc:287] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.545s user 0.014s sys 0.000s
I20260807 01:47:57.324231 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:57.331461 31179 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:57.332357 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:57.312207 31255 consensus_queue.cc:260] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.306555 28702 raft_consensus.cc:2468] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:57.334741 31329 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:57.338505 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:57.339272 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:57.341467 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:47:57.341724 31336 log.cc:927] Time spent T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.107s user 0.001s sys 0.000s
I20260807 01:47:57.320981 31254 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:57.391602 31254 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.393884 31254 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:57.342721 31192 raft_consensus.cc:2804] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader election won for term 3
W20260807 01:47:57.346704 31357 log.cc:927] Time spent T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.087s user 0.000s sys 0.001s
I20260807 01:47:57.348747 31203 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:57.394893 27175 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:57.348544 31207 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:57.355180 28708 raft_consensus.cc:1240] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 3. Current term is 4. Ops: []
I20260807 01:47:57.355535 28707 raft_consensus.cc:1275] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260807 01:47:57.356534 28660 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:57.374128 31122 raft_consensus.cc:2764] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260807 01:47:57.402944 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:57.404314 31207 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:57.370611 27172 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:57.374713 27227 raft_consensus.cc:2393] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 8: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:57.375602 31161 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.378786 28710 raft_consensus.cc:2393] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 9: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:47:57.377241 27104 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 10.
I20260807 01:47:57.382251 31289 raft_consensus.cc:2804] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:47:57.425088 31289 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:57.401926 31213 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:57.343158 31322 raft_consensus.cc:515] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:57.425954 31356 log.cc:927] Time spent T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.181s user 0.003s sys 0.000s
I20260807 01:47:57.409304 31192 raft_consensus.cc:697] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:57.383558 31255 raft_consensus.cc:3060] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
W20260807 01:47:57.361631 31359 log.cc:927] Time spent T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.091s user 0.000s sys 0.002s
I20260807 01:47:57.384068 28713 raft_consensus.cc:1275] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260807 01:47:57.385638 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:57.370229 31290 consensus_queue.cc:1048] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260807 01:47:57.390206 31189 consensus_queue.cc:1048] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260807 01:47:57.393886 27326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33976 (request call id 2509) took 1634 ms. Trace:
W20260807 01:47:57.396612 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:57.396699 27214 raft_consensus.cc:1240] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 4. Current term is 5. Ops: []
I20260807 01:47:57.407703 27212 raft_consensus.cc:1275] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 01:47:57.425642 31289 raft_consensus.cc:3060] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:47:57.383103 27229 raft_consensus.cc:1275] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260807 01:47:57.382480 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:47:57.429605 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 59072.3us
I20260807 01:47:57.430135 31244 raft_consensus.cc:2804] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader election won for term 8
I20260807 01:47:57.433578 27287 leader_election.cc:304] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:57.439915 27067 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:57.444420 27172 leader_election.cc:304] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:57.399247 31003 consensus_queue.cc:1059] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260807 01:47:57.434373 31207 raft_consensus.cc:2804] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:57.425537 31161 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:47:57.439828 27326 rpcz_store.cc:276] 0807 01:47:55.759109 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:55.759445 (+ 336us) service_pool.cc:224] Handling call
0807 01:47:56.524034 (+764589us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:47:56.668853 (+144819us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:47:56.668884 (+ 31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:47:56.705097 (+ 36213us) log.cc:844] Serialized 26 byte log entry
0807 01:47:56.875361 (+170264us) raft_consensus.cc:1645] Marking committed up to 2
0807 01:47:56.875405 (+ 44us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:56.875470 (+ 65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:47:57.393524 (+518054us) raft_consensus.cc:1685] finished
0807 01:47:57.393544 (+ 20us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:47:57.393693 (+ 149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4310,"raft.queue_time_us":24688,"raft.run_cpu_time_us":755,"raft.run_wall_time_us":77556,"thread_start_us":116431,"threads_started":1,"wal-append.queue_time_us":371728}
I20260807 01:47:57.447505 31192 consensus_queue.cc:237] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.449502 28709 raft_consensus.cc:2468] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 1.
I20260807 01:47:57.450945 31254 raft_consensus.cc:1064] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:57.452975 30681 raft_consensus.cc:3060] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:57.453522 31272 raft_consensus.cc:2804] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:47:57.452438 27067 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:57.455260 31329 raft_consensus.cc:2804] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:57.455703 31244 raft_consensus.cc:697] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 8 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:57.456250 30742 raft_consensus.cc:2804] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:47:57.457436 30888 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.458657 31204 raft_consensus.cc:1064] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:47:57.462276 27172 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:57.463967 27175 leader_election.cc:304] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
W20260807 01:47:57.465227 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 108842us
W20260807 01:47:57.466389 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 56296.3us
I20260807 01:47:57.428009 31213 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.470719 27172 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
W20260807 01:47:57.476432 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:57.473908 31353 raft_consensus.cc:2804] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:57.477960 31353 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:57.478379 31353 raft_consensus.cc:3060] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:57.464478 31272 raft_consensus.cc:493] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:47:57.470881 31366 log.cc:927] Time spent T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.057s user 0.000s sys 0.009s
I20260807 01:47:57.465876 31329 raft_consensus.cc:493] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
W20260807 01:47:57.481985 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:57.483491 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:57.483664 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:57.470222 31207 raft_consensus.cc:697] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:57.500526 31207 consensus_queue.cc:237] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.501533 31187 consensus_queue.cc:1059] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 01:47:57.504331 31298 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:57.482704 31272 raft_consensus.cc:3060] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:57.488920 31259 raft_consensus.cc:3055] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:57.480988 31203 raft_consensus.cc:2804] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260807 01:47:57.476182 31269 raft_consensus.cc:2749] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:47:57.490628 31053 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:57.482357 30742 raft_consensus.cc:697] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 6 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:57.491503 31244 consensus_queue.cc:237] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:57.476377 31102 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:57.573976 31370 raft_consensus.cc:493] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:57.574609 31370 raft_consensus.cc:515] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.574937 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:57.575847 28711 raft_consensus.cc:2410] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260807 01:47:57.576870 31370 leader_election.cc:290] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:57.497581 31306 raft_consensus.cc:3060] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:57.485687 31329 raft_consensus.cc:3060] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:57.501041 31322 leader_election.cc:290] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:47:57.486384 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 52928.4us
I20260807 01:47:57.536090 31254 consensus_queue.cc:237] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.537288 31353 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.538357 31213 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:57.540695 30681 raft_consensus.cc:2749] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260807 01:47:57.545917 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:57.576941 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
W20260807 01:47:57.552614 27174 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:57.556123 31271 raft_consensus.cc:3055] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:57.547523 27064 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:57.531998 30888 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:57.564067 28715 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Advancing to term 4
I20260807 01:47:57.577761 28709 raft_consensus.cc:1275] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260807 01:47:57.578061 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 93951us
I20260807 01:47:57.490911 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:47:57.564921 31203 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:57.565469 31259 raft_consensus.cc:740] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:57.559677 31298 consensus_queue.cc:1048] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
W20260807 01:47:57.548281 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 61368.5us
W20260807 01:47:57.595409 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 89294.5us
I20260807 01:47:57.597878 30854 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:57.579602 31122 consensus_queue.cc:1059] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260807 01:47:57.609992 27287 leader_election.cc:304] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:57.594785 28713 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Advancing to term 10
I20260807 01:47:57.590544 31053 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:57.625205 27063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:57.625487 31223 raft_consensus.cc:3055] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:47:57.625785 31203 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 10 FOLLOWER]: Advancing to term 11
I20260807 01:47:57.595033 31271 raft_consensus.cc:740] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:57.626768 28660 leader_election.cc:304] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:47:57.626432 30742 consensus_queue.cc:237] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:57.627329 28661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:57.607365 31213 raft_consensus.cc:2804] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:47:57.632751 31194 consensus_queue.cc:1048] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260807 01:47:57.635084 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:47:57.641958 28655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:57.603195 31189 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:57.655257 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:57.659718 26952 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46152 (request call id 389) took 10538 ms. Trace:
I20260807 01:47:57.660862 31259 consensus_queue.cc:260] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.662052 31259 raft_consensus.cc:3060] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:57.662372 31053 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:57.662751 28709 raft_consensus.cc:1240] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 3. Current term is 4. Ops: []
I20260807 01:47:57.663200 31189 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.637741 31272 raft_consensus.cc:515] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.666327 31281 raft_consensus.cc:2749] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 01:47:57.636595 30888 consensus_queue.cc:1048] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260807 01:47:57.668135 27337 raft_consensus.cc:2468] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:47:57.669283 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:57.672211 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:57.652567 31213 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:57.672438 31204 consensus_queue.cc:237] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.673216 31213 raft_consensus.cc:3060] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:57.674422 31289 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:57.641049 31370 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:57.676971 31370 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.679055 31370 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:57.646668 31176 consensus_queue.cc:1048] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260807 01:47:57.679886 31192 raft_consensus.cc:3055] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:57.680334 31192 raft_consensus.cc:740] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:57.680578 31053 raft_consensus.cc:493] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:57.681142 31053 raft_consensus.cc:515] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:57.683012 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 56371.4us
I20260807 01:47:57.683378 31053 leader_election.cc:290] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:57.638310 31187 consensus_queue.cc:1048] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260807 01:47:57.683941 26948 catalog_manager.cc:5697] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 6 to 7, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4). New cstate: current_term: 7 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:57.685730 31271 consensus_queue.cc:260] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:57.687199 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:57.687850 27214 raft_consensus.cc:2376] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4. Re-sending same reply.
W20260807 01:47:57.688416 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:57.688668 28713 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 10.
I20260807 01:47:57.672629 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:57.690857 27289 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:57.691732 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:57.691929 31353 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:57.688916 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:57.694515 27230 raft_consensus.cc:2376] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5. Re-sending same reply.
I20260807 01:47:57.673007 28708 raft_consensus.cc:2468] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:47:57.695546 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:47:57.696126 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 86299.6us
I20260807 01:47:57.696290 28709 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:57.672246 31179 raft_consensus.cc:2804] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:57.696286 31192 consensus_queue.cc:260] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.695537 26948 catalog_manager.cc:5697] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 5 to 6, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4). New cstate: current_term: 6 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:47:57.641053 31223 raft_consensus.cc:740] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:57.660183 26952 rpcz_store.cc:276] 0807 01:47:47.120948 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:47.131601 (+ 10653us) service_pool.cc:224] Handling call
0807 01:47:57.659622 (+10528021us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 01:47:57.219287 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 01:47:57.220002 (+ 715us) write_op.cc:432] Acquiring schema lock in shared mode
0807 01:47:57.220059 (+ 57us) write_op.cc:435] Acquired schema lock
0807 01:47:57.220071 (+ 12us) tablet.cc:662] Decoding operations
0807 01:47:57.220500 (+ 429us) write_op.cc:620] Acquiring the partition lock for write op
0807 01:47:57.220592 (+ 92us) write_op.cc:641] Partition lock acquired for write op
0807 01:47:57.220621 (+ 29us) tablet.cc:685] Acquiring locks for 1 operations
0807 01:47:57.220951 (+ 330us) tablet.cc:701] Row locks acquired
0807 01:47:57.220971 (+ 20us) write_op.cc:260] PREPARE: finished
0807 01:47:57.221171 (+ 200us) write_op.cc:270] Start()
0807 01:47:57.221338 (+ 167us) write_op.cc:276] Timestamp: P: 1786067277221092 usec, L: 0
0807 01:47:57.221359 (+ 21us) op_driver.cc:348] REPLICATION: starting
0807 01:47:57.222185 (+ 826us) log.cc:844] Serialized 527 byte log entry
0807 01:47:57.424000 (+201815us) spinlock_profiling.cc:243] Waited 65.6 ms on lock 0x7b54002afe20. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55388470 00007f7d55395a98 00007f7d553956da 00007f7d55419d45 00007f7d5541a1e9 00007f7d58e73f61 00007f7d58e73738 00007f7d58e783d1 00007f7d58e78389 00007f7d58e78319 00007f7d58e782e1
0807 01:47:57.440541 (+ 16541us) op_driver.cc:464] REPLICATION: finished
0807 01:47:57.450776 (+ 10235us) write_op.cc:301] APPLY: starting
0807 01:47:57.450864 (+ 88us) tablet.cc:1367] starting BulkCheckPresence
0807 01:47:57.450973 (+ 109us) tablet.cc:1370] finished BulkCheckPresence
0807 01:47:57.450993 (+ 20us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 01:47:57.451406 (+ 413us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 01:47:57.451589 (+ 183us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0807 01:47:57.451635 (+ 46us) write_op.cc:312] APPLY: finished
0807 01:47:57.451833 (+ 198us) log.cc:844] Serialized 23 byte log entry
0807 01:47:57.452005 (+ 172us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:47:57.452088 (+ 83us) write_op.cc:454] Released schema lock
0807 01:47:57.452130 (+ 42us) write_op.cc:341] FINISH: Updating metrics
0807 01:47:57.452493 (+ 363us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:47:57.452526 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3846,"reported_tablets":8,"spinlock_wait_cycles":1920,"child_traces":[["op",{"apply.queue_time_us":3823,"apply.run_cpu_time_us":1696,"apply.run_wall_time_us":1697,"num_ops":1,"prepare.queue_time_us":130762,"prepare.run_cpu_time_us":7583,"prepare.run_wall_time_us":205079,"replication_time_us":206834,"spinlock_wait_cycles":150875520,"thread_start_us":51050,"threads_started":1}]]}
I20260807 01:47:57.700167 31192 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:57.699536 27175 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
W20260807 01:47:57.703132 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 63873.5us
I20260807 01:47:57.706439 27067 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:57.709039 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:57.709899 27230 raft_consensus.cc:2368] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for earlier term 2. Current term is 3.
I20260807 01:47:57.711845 28655 leader_election.cc:304] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:57.714390 28715 raft_consensus.cc:1275] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260807 01:47:57.714911 28715 pending_rounds.cc:85] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12: Aborting all ops after (but not including) 2
I20260807 01:47:57.701277 26948 catalog_manager.cc:5697] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 4 to 6, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 6 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:57.715201 28715 pending_rounds.cc:107] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12: Aborting uncommitted NO_OP operation due to leader change: 3.3
W20260807 01:47:57.742377 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 44227.2us
I20260807 01:47:57.742615 27067 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:57.743778 28715 raft_consensus.cc:2889] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 01:47:57.744345 27067 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:57.694389 30888 raft_consensus.cc:2804] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Leader election won for term 10
I20260807 01:47:57.749478 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:57.750141 28709 raft_consensus.cc:3060] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 1 FOLLOWER]: Advancing to term 3
I20260807 01:47:57.752285 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:57.759296 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:57.760071 27105 raft_consensus.cc:2468] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:47:57.762684 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:57.748080 31090 consensus_queue.cc:1059] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.016s
I20260807 01:47:57.763813 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:57.680358 31329 raft_consensus.cc:515] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.766613 31329 leader_election.cc:290] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:57.766299 31223 consensus_queue.cc:260] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.767586 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:57.768280 27214 raft_consensus.cc:2468] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:47:57.768357 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:57.768707 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:57.769043 30888 raft_consensus.cc:697] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 10 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:47:57.770207 30888 consensus_queue.cc:237] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:57.771737 31272 leader_election.cc:290] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:57.687211 31271 raft_consensus.cc:3060] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:57.775146 27230 raft_consensus.cc:2393] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 3: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:47:57.698061 31179 raft_consensus.cc:697] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:57.745277 31370 raft_consensus.cc:2804] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:47:57.757334 26948 catalog_manager.cc:5697] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:47:57.734076 31053 raft_consensus.cc:2804] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:47:57.775908 27099 raft_consensus.cc:2468] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:47:57.787957 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:57.798369 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:57.799345 28710 raft_consensus.cc:2393] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 3: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:47:57.767913 31223 raft_consensus.cc:3060] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:57.755400 31176 raft_consensus.cc:2749] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:57.809528 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:57.780258 31194 raft_consensus.cc:2804] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260807 01:47:57.818512 27064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:57.774413 30681 raft_consensus.cc:2749] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:47:57.825223 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:47:57.826107 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:57.811244 27172 leader_election.cc:304] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:57.795234 31298 raft_consensus.cc:3055] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:57.789641 31370 raft_consensus.cc:493] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:57.829856 27172 leader_election.cc:304] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:57.755791 31235 consensus_queue.cc:1048] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260807 01:47:57.799336 31053 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:57.839272 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:57.782123 31213 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:57.814033 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:57.833544 27100 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
W20260807 01:47:57.860258 28660 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:57.825286 31194 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:57.863387 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:57.866652 27108 raft_consensus.cc:2468] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:47:57.821220 31289 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:57.832540 31353 raft_consensus.cc:2804] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:47:57.838914 31370 raft_consensus.cc:3060] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:57.839129 31053 raft_consensus.cc:3060] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:57.829371 31298 raft_consensus.cc:740] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:57.858609 31150 raft_consensus.cc:2804] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260807 01:47:57.879573 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 81276.1us
W20260807 01:47:57.883806 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:47:57.887769 28662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:57.864130 31269 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:57.894189 31102 raft_consensus.cc:2749] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:57.894649 31179 consensus_queue.cc:237] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:57.895704 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 63486.2us
I20260807 01:47:57.892233 31298 consensus_queue.cc:260] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:57.923189 27213 raft_consensus.cc:1275] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 01:47:57.908155 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:57.924801 27222 raft_consensus.cc:3060] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:57.872612 31194 raft_consensus.cc:3060] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
W20260807 01:47:57.915887 27064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:57.877725 31353 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:57.934810 28706 raft_consensus.cc:1275] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEARNER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 01:47:57.937415 31353 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:57.902974 31150 raft_consensus.cc:493] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:57.947572 31150 raft_consensus.cc:3060] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:57.949386 27227 raft_consensus.cc:1275] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260807 01:47:57.950939 31298 raft_consensus.cc:3060] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:57.975128 27287 leader_election.cc:304] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
W20260807 01:47:57.980237 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:57.986208 28709 raft_consensus.cc:2468] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:47:57.983476 31235 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:58.003988 31329 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:58.008020 31329 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:58.009565 31003 raft_consensus.cc:2804] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 01:47:58.010003 31003 raft_consensus.cc:493] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:58.010421 31003 raft_consensus.cc:3060] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:58.010522 31329 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:47:58.013725 28660 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:57.978567 31269 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:58.002341 31204 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:47:58.030895 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 55881.8us
I20260807 01:47:58.032347 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:58.037894 28709 raft_consensus.cc:1275] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [term 6 LEARNER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260807 01:47:58.042806 28660 leader_election.cc:304] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
W20260807 01:47:58.083318 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 43982.5us
I20260807 01:47:58.017272 31235 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:58.085567 27099 raft_consensus.cc:2468] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:47:58.084992 27222 raft_consensus.cc:2468] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:47:58.084165 28660 leader_election.cc:304] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:47:58.086966 27172 leader_election.cc:304] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:58.029629 31389 consensus_queue.cc:1048] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260807 01:47:58.084818 31385 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:47:58.106621 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:58.091475 31397 raft_consensus.cc:2804] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:47:58.083920 31270 consensus_queue.cc:1048] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260807 01:47:58.110327 28660 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:47:58.097991 31161 raft_consensus.cc:2804] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:47:58.098021 30854 raft_consensus.cc:2804] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:47:58.113482 27287 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:58.130607 31281 raft_consensus.cc:2804] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:47:58.131038 31281 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:58.131454 31281 raft_consensus.cc:3060] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:58.124094 31397 raft_consensus.cc:493] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:58.132051 31397 raft_consensus.cc:3060] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:58.131138 27287 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:58.136765 31255 raft_consensus.cc:2804] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:47:58.137205 31255 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:58.137584 31255 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
W20260807 01:47:58.128011 31402 log.cc:927] Time spent T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.080s user 0.000s sys 0.001s
I20260807 01:47:58.140107 31150 raft_consensus.cc:515] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:58.143405 27287 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:58.143810 31150 leader_election.cc:290] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:58.119952 31290 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:58.153852 31255 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:58.154548 31290 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:58.156471 31255 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:58.156275 31370 raft_consensus.cc:515] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:58.156715 31290 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:58.106395 31053 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:58.158816 31053 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:47:58.122891 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:58.082732 31204 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:58.108479 31385 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:58.128053 31203 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:58.137967 31259 consensus_queue.cc:1048] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
W20260807 01:47:58.148761 31405 log.cc:927] Time spent T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.051s user 0.000s sys 0.000s
I20260807 01:47:58.151269 27287 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:58.151867 31271 raft_consensus.cc:2804] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260807 01:47:58.161471 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 51228.7us
I20260807 01:47:58.162933 27102 raft_consensus.cc:2468] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:58.167361 27064 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:58.114434 31269 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:47:58.175194 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 88293.8us
I20260807 01:47:58.173719 31161 raft_consensus.cc:697] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
W20260807 01:47:58.176868 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:47:58.189549 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:58.181814 31189 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:58.190800 31394 consensus_queue.cc:1048] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:47:58.192260 31409 raft_consensus.cc:2804] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:47:58.192783 31409 raft_consensus.cc:697] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:47:58.193821 31409 consensus_queue.cc:237] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:58.194517 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:58.174281 31306 raft_consensus.cc:2804] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:47:58.195237 27104 raft_consensus.cc:2468] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 2.
I20260807 01:47:58.202328 27212 raft_consensus.cc:1275] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 01:47:58.204730 27172 leader_election.cc:304] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:58.206203 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:58.206516 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:58.207154 27110 raft_consensus.cc:3060] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:58.208550 28660 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:47:58.209723 27222 raft_consensus.cc:1275] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260807 01:47:58.181738 31271 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:58.176385 31389 consensus_queue.cc:1048] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.026s
I20260807 01:47:58.176447 31353 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:58.236549 31353 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:58.198776 31370 leader_election.cc:290] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:58.222091 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:47:58.225706 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:58.221498 27216 raft_consensus.cc:1240] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 3. Current term is 4. Ops: []
I20260807 01:47:58.190562 31102 consensus_queue.cc:1048] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260807 01:47:58.229928 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:47:58.230381 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 79184.7us
I20260807 01:47:58.189814 30854 raft_consensus.cc:697] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:58.198453 31306 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:58.195967 31255 raft_consensus.cc:2749] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:47:58.268548 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:58.178619 31194 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:58.269284 27337 raft_consensus.cc:3060] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:58.266599 27104 raft_consensus.cc:3060] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
W20260807 01:47:58.295835 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 94901.8us
I20260807 01:47:58.308247 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
W20260807 01:47:58.278184 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 50579.6us
I20260807 01:47:58.323271 27104 raft_consensus.cc:2468] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:58.323297 27287 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:58.321544 28660 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:47:58.313184 27175 leader_election.cc:304] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:47:58.321182 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:58.249336 27326 raft_consensus.cc:1240] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 4. Current term is 5. Ops: []
I20260807 01:47:58.222579 31053 raft_consensus.cc:2804] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260807 01:47:58.273077 31419 log.cc:927] Time spent T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.069s user 0.004s sys 0.000s
W20260807 01:47:58.338821 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 44020.8us
W20260807 01:47:58.340113 27063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:47:58.340636 27063 leader_election.cc:336] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.700s (SENT)
I20260807 01:47:58.340930 27063 leader_election.cc:304] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:58.221153 31271 raft_consensus.cc:3060] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:58.343420 27100 raft_consensus.cc:2393] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 4: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:47:58.236893 31416 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:58.344827 28708 raft_consensus.cc:1275] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 01:47:58.330024 31053 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:47:58.347718 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:58.293241 31370 raft_consensus.cc:2804] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:58.350867 27337 raft_consensus.cc:2468] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:47:58.276268 31235 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:58.327493 31270 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:58.273277 31306 raft_consensus.cc:3060] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:58.329481 31329 raft_consensus.cc:2804] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:47:58.320622 31161 consensus_queue.cc:237] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:58.336321 31385 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:58.354460 31416 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:58.369817 31397 raft_consensus.cc:515] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:58.362313 31269 consensus_queue.cc:1048] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260807 01:47:58.355048 31053 raft_consensus.cc:3060] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
W20260807 01:47:58.388585 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 67123.9us
I20260807 01:47:58.374384 31270 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:58.375434 31189 raft_consensus.cc:2804] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader election won for term 5
W20260807 01:47:58.378736 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:58.337651 31203 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 11 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:58.361733 31187 raft_consensus.cc:2749] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:58.300804 31204 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:58.377286 31003 raft_consensus.cc:515] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:58.371812 30854 consensus_queue.cc:237] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:58.348186 31255 raft_consensus.cc:2749] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 01:47:58.408247 27110 raft_consensus.cc:2468] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:47:58.388535 31370 raft_consensus.cc:697] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:58.448350 28660 leader_election.cc:400] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for earlier term 2. Current term is 3.
I20260807 01:47:58.448843 28660 leader_election.cc:403] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260807 01:47:58.444236 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 103626us
I20260807 01:47:58.375468 31281 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
W20260807 01:47:58.454566 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 211102us
I20260807 01:47:58.421446 31329 raft_consensus.cc:697] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
W20260807 01:47:58.460510 27063 leader_election.cc:336] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.795s (SENT)
I20260807 01:47:58.461045 27063 leader_election.cc:304] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:58.452394 31259 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:58.482640 31259 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:58.485112 31259 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:58.486821 31122 raft_consensus.cc:2749] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for earlier term 2. Current term is 3.
I20260807 01:47:58.443413 30742 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:58.487882 30742 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:58.492424 31003 leader_election.cc:290] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:58.458297 31189 raft_consensus.cc:697] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
W20260807 01:47:58.494464 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 72369.2us
I20260807 01:47:58.495309 28660 leader_election.cc:304] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:58.496277 31409 consensus_queue.cc:1048] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260807 01:47:58.499629 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:47:58.504251 27064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:58.505678 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:58.505918 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:58.506394 27100 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:58.506604 27110 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Advancing to term 11
I20260807 01:47:58.514971 27104 raft_consensus.cc:1275] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260807 01:47:58.518751 27064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:58.541464 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:58.541915 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:58.542915 27172 leader_election.cc:304] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:58.501231 31416 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:58.544576 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:58.544783 31270 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:58.544947 28707 raft_consensus.cc:1275] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 10 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260807 01:47:58.544122 27172 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:58.545818 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:58.546183 31416 raft_consensus.cc:2804] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:47:58.546634 31416 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:58.546969 31416 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:58.547521 27102 raft_consensus.cc:1275] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 01:47:58.548418 31171 raft_consensus.cc:2749] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 01:47:58.490047 30742 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:58.549942 28655 leader_election.cc:304] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:58.550673 30742 consensus_queue.cc:1059] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 01:47:58.551844 30742 raft_consensus.cc:3055] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:58.552186 30742 raft_consensus.cc:740] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
W20260807 01:47:58.553639 31431 log.cc:927] Time spent T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.084s user 0.002s sys 0.000s
W20260807 01:47:58.555485 31429 log.cc:927] Time spent T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.093s user 0.000s sys 0.000s
I20260807 01:47:58.443663 31290 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:58.557062 31290 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:58.556119 31416 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:47:58.559216 31290 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:58.481823 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:58.560467 31194 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:58.561662 31416 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:47:58.562261 31387 log.cc:927] Time spent T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.076s user 0.004s sys 0.000s
I20260807 01:47:58.562736 31189 consensus_queue.cc:237] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:58.564797 31385 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:58.545230 27108 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:47:58.565299 31385 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:58.566110 31391 log.cc:927] Time spent T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.080s user 0.000s sys 0.004s
W20260807 01:47:58.567049 31298 consensus_meta.cc:287] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.615s user 0.014s sys 0.000s
I20260807 01:47:58.567425 31385 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:58.568004 31407 raft_consensus.cc:2804] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:58.568583 31407 raft_consensus.cc:697] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:58.568873 31281 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:58.569442 31407 consensus_queue.cc:237] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:58.569783 31329 consensus_queue.cc:237] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:58.561687 27102 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:41078
I20260807 01:47:58.573374 27102 raft_consensus.cc:606] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Received request to transfer leadership
I20260807 01:47:58.572654 27110 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 11.
I20260807 01:47:58.543967 31353 raft_consensus.cc:2804] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:47:58.573083 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:47:58.584988 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:58.585826 27328 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:47:58.587409 27337 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
W20260807 01:47:58.588686 31423 log.cc:927] Time spent T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.126s user 0.000s sys 0.003s
I20260807 01:47:58.575515 27216 raft_consensus.cc:2393] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 5: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
W20260807 01:47:58.562222 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 101747us
I20260807 01:47:58.553133 30742 consensus_queue.cc:260] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:58.591377 30742 raft_consensus.cc:3060] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:58.593468 31213 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:58.576145 27100 raft_consensus.cc:2468] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:47:58.575469 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:58.588199 31053 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:58.617388 26948 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60866 (request call id 217) took 1405 ms. Trace:
I20260807 01:47:58.553472 31397 leader_election.cc:290] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:58.562829 31426 raft_consensus.cc:2749] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:58.541884 31204 consensus_queue.cc:1048] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.096s
I20260807 01:47:58.619645 31353 raft_consensus.cc:697] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:58.618048 26948 rpcz_store.cc:276] 0807 01:47:57.177121 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:57.177448 (+ 327us) service_pool.cc:224] Handling call
0807 01:47:57.432973 (+255525us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b54002ffdd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55388470 00007f7d5542731a 00007f7d5b21738c 00007f7d5b313bfe 00007f7d597cd4a4 00007f7d597cd432 00007f7d597cd361 00007f7d597cd2dc 00007f7d597cc592 00007f7d5375807c 00007f7d537574b6
0807 01:47:58.582465 (+1149492us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 01:47:57.898229 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 01:47:57.898804 (+ 575us) write_op.cc:432] Acquiring schema lock in shared mode
0807 01:47:57.898840 (+ 36us) write_op.cc:435] Acquired schema lock
0807 01:47:57.898857 (+ 17us) tablet.cc:662] Decoding operations
0807 01:47:57.899453 (+ 596us) write_op.cc:620] Acquiring the partition lock for write op
0807 01:47:57.899512 (+ 59us) write_op.cc:641] Partition lock acquired for write op
0807 01:47:57.899538 (+ 26us) tablet.cc:685] Acquiring locks for 4 operations
0807 01:47:57.899928 (+ 390us) tablet.cc:701] Row locks acquired
0807 01:47:57.899941 (+ 13us) write_op.cc:260] PREPARE: finished
0807 01:47:57.900104 (+ 163us) write_op.cc:270] Start()
0807 01:47:57.900245 (+ 141us) write_op.cc:276] Timestamp: P: 1786067277900048 usec, L: 0
0807 01:47:57.900261 (+ 16us) op_driver.cc:348] REPLICATION: starting
0807 01:47:57.900908 (+ 647us) log.cc:844] Serialized 1770 byte log entry
0807 01:47:57.991478 (+ 90570us) op_driver.cc:464] REPLICATION: finished
0807 01:47:58.217475 (+225997us) write_op.cc:301] APPLY: starting
0807 01:47:58.227149 (+ 9674us) tablet.cc:1367] starting BulkCheckPresence
0807 01:47:58.238613 (+ 11464us) tablet.cc:1370] finished BulkCheckPresence
0807 01:47:58.238789 (+ 176us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 01:47:58.239600 (+ 811us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 01:47:58.239904 (+ 304us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0807 01:47:58.239956 (+ 52us) write_op.cc:312] APPLY: finished
0807 01:47:58.310933 (+ 70977us) log.cc:844] Serialized 41 byte log entry
0807 01:47:58.337554 (+ 26621us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:47:58.350717 (+ 13163us) write_op.cc:454] Released schema lock
0807 01:47:58.355800 (+ 5083us) write_op.cc:341] FINISH: Updating metrics
0807 01:47:58.433105 (+ 77305us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:47:58.437806 (+ 4701us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":14,"spinlock_wait_cycles":3806976,"child_traces":[["op",{"apply.queue_time_us":202715,"apply.run_cpu_time_us":3880,"apply.run_wall_time_us":202997,"num_ops":4,"prepare.queue_time_us":2116,"prepare.run_cpu_time_us":3022,"prepare.run_wall_time_us":23530,"replication_time_us":78380,"thread_start_us":90072,"threads_started":1}]]}
W20260807 01:47:58.575260 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 170386us
W20260807 01:47:58.702302 28662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:58.581853 31281 consensus_queue.cc:1048] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.089s
W20260807 01:47:58.731954 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:58.561985 31370 consensus_queue.cc:237] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:58.773684 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 71543.2us
I20260807 01:47:58.778687 31290 raft_consensus.cc:493] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:58.779299 31290 raft_consensus.cc:515] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:58.781482 31290 leader_election.cc:290] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:47:58.782315 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:58.783046 30888 raft_consensus.cc:1064] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
W20260807 01:47:58.783532 30888 raft_consensus.cc:1068] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade: Unable to promote non-voter 57bb4985157447f0a3ab680fa0026f12: Illegal state: Leader has not yet committed an operation in its own term
I20260807 01:47:58.785166 27100 raft_consensus.cc:1240] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 3. Current term is 5. Ops: []
W20260807 01:47:58.785004 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 289755us
I20260807 01:47:58.786346 28660 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:47:58.787077 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:58.787232 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:47:58.788514 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:58.803292 28660 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:58.803499 28655 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:58.838444 28655 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:58.840684 31329 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:58.841363 31329 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:58.840710 31353 consensus_queue.cc:237] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:58.849134 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 45940.8us
I20260807 01:47:58.854285 27100 raft_consensus.cc:1240] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 8 LEADER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 7. Current term is 8. Ops: []
I20260807 01:47:58.856683 27100 raft_consensus.cc:1240] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 6. Current term is 7. Ops: []
I20260807 01:47:58.856966 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:58.857870 28709 raft_consensus.cc:2393] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 4: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:47:58.850296 28660 leader_election.cc:304] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:58.861806 27108 raft_consensus.cc:1240] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5. Ops: []
I20260807 01:47:58.861611 31053 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:58.849967 31171 raft_consensus.cc:2804] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:47:58.867173 31171 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:58.867568 31171 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:58.867753 27099 raft_consensus.cc:1240] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5. Ops: []
I20260807 01:47:58.838199 31270 raft_consensus.cc:2749] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:47:58.809965 31122 raft_consensus.cc:2804] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:47:58.874825 27230 raft_consensus.cc:1275] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260807 01:47:58.875187 27230 pending_rounds.cc:85] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f: Aborting all ops after (but not including) 1
I20260807 01:47:58.875404 27230 pending_rounds.cc:107] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260807 01:47:58.875619 27230 raft_consensus.cc:2889] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 01:47:58.875602 27110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:41078
I20260807 01:47:58.876579 27110 raft_consensus.cc:606] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Received request to transfer leadership
I20260807 01:47:58.877092 27212 raft_consensus.cc:1240] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 7. Current term is 8. Ops: []
I20260807 01:47:58.873939 31122 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:58.879223 31122 raft_consensus.cc:3060] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:58.880486 28660 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:47:58.880322 27102 raft_consensus.cc:1240] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3. Ops: []
I20260807 01:47:58.883030 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:58.883688 27102 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 8.
I20260807 01:47:58.858593 31397 raft_consensus.cc:2804] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:47:58.886698 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:58.887629 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:58.897976 27102 raft_consensus.cc:1275] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:47:58.897644 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:58.899132 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:58.899537 27099 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Advancing to term 10
I20260807 01:47:58.902727 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:58.902501 31053 consensus_queue.cc:1059] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260807 01:47:58.905061 31053 raft_consensus.cc:3055] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:47:58.905429 31053 raft_consensus.cc:740] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:58.906453 27229 raft_consensus.cc:1240] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 6. Current term is 7. Ops: []
I20260807 01:47:58.877230 31117 raft_consensus.cc:2749] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:47:58.885468 31270 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:58.911394 27172 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:58.912649 27105 raft_consensus.cc:2468] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:47:58.890734 31407 raft_consensus.cc:2749] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:47:58.891223 31269 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:58.915563 27172 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:58.916409 27172 leader_election.cc:304] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:47:58.917435 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:58.895881 31397 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:58.917940 27105 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:58.918795 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:58.919039 31420 raft_consensus.cc:2804] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 11 FOLLOWER]: Leader election won for term 11
I20260807 01:47:58.919471 31203 raft_consensus.cc:2804] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:47:58.947434 27227 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:47:58.914417 31416 consensus_queue.cc:1048] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260807 01:47:58.920872 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:58.964679 27101 raft_consensus.cc:2468] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:58.912869 31207 consensus_queue.cc:1048] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260807 01:47:58.924245 31150 raft_consensus.cc:2804] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:47:58.922139 31329 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:58.924968 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:58.989334 27102 raft_consensus.cc:3060] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:58.956879 27099 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 10.
I20260807 01:47:58.993395 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:58.953118 31397 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Advancing to term 7
W20260807 01:47:58.967290 31306 consensus_meta.cc:287] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 0.560s user 0.009s sys 0.002s
I20260807 01:47:58.999373 27064 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:59.000361 27100 raft_consensus.cc:1275] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260807 01:47:58.994879 31161 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s user 0.011s sys 0.016s
I20260807 01:47:58.924121 31122 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:59.009382 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:58.987722 31150 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:47:59.014051 27229 raft_consensus.cc:1240] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260807 01:47:58.952059 31269 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:59.010569 31161 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.530s user 0.011s sys 0.016s
I20260807 01:47:58.968734 31270 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:59.025902 27170 leader_election.cc:336] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.574s (SENT)
I20260807 01:47:59.027832 27170 leader_election.cc:304] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:59.008899 31090 raft_consensus.cc:2804] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260807 01:47:59.034243 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 103124us
I20260807 01:47:59.035163 27212 raft_consensus.cc:1240] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5. Ops: []
I20260807 01:47:59.015650 31150 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:59.035430 27172 leader_election.cc:304] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:47:59.008392 31306 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:59.062614 27102 raft_consensus.cc:2468] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:47:59.034744 31090 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 01:47:59.064087 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 187111us
I20260807 01:47:59.067876 27172 leader_election.cc:304] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
W20260807 01:47:59.068380 27170 leader_election.cc:336] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.616s (SENT)
I20260807 01:47:59.069103 27170 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:59.072968 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:59.078437 27212 raft_consensus.cc:3060] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:59.063699 31353 raft_consensus.cc:2804] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260807 01:47:59.079524 26932 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 96624.9us
I20260807 01:47:59.079856 31353 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:59.080323 31353 raft_consensus.cc:3060] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:59.081437 31270 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:59.091069 31122 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:47:59.056572 31207 raft_consensus.cc:2749] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:47:59.088835 31353 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:59.096817 31053 consensus_queue.cc:260] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:59.098870 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 234455us
W20260807 01:47:59.108824 27064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:47:59.109560 27063 leader_election.cc:336] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.502s (SENT)
I20260807 01:47:59.110204 27063 leader_election.cc:304] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:59.070605 31090 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:59.115614 27105 raft_consensus.cc:2468] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:47:59.117233 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:59.075860 31329 raft_consensus.cc:2804] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:47:59.122637 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:47:59.121093 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 144173us
I20260807 01:47:59.124748 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:59.130318 31420 raft_consensus.cc:697] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 11 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
W20260807 01:47:59.130792 31440 log.cc:927] Time spent T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.120s user 0.000s sys 0.001s
W20260807 01:47:59.131913 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 73858.9us
I20260807 01:47:59.132249 27230 raft_consensus.cc:2393] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 5: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
W20260807 01:47:59.133090 27170 leader_election.cc:336] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.639s (SENT)
I20260807 01:47:59.133639 27170 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:59.118673 31416 raft_consensus.cc:2749] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:47:59.120472 28713 raft_consensus.cc:1240] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5. Ops: []
I20260807 01:47:59.150094 27218 raft_consensus.cc:1240] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5. Ops: []
I20260807 01:47:59.113520 31194 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:47:59.140810 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:59.143239 27102 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:41078
I20260807 01:47:59.152187 27326 raft_consensus.cc:1275] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260807 01:47:59.144887 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:47:59.154815 27102 raft_consensus.cc:606] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Received request to transfer leadership
I20260807 01:47:59.154737 27326 pending_rounds.cc:85] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade: Aborting all ops after (but not including) 6
I20260807 01:47:59.159471 27326 pending_rounds.cc:107] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260807 01:47:59.159803 27326 raft_consensus.cc:2889] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260807 01:47:59.158399 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 261875us
I20260807 01:47:59.169389 27212 raft_consensus.cc:2468] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:47:59.157030 27227 raft_consensus.cc:2468] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
W20260807 01:47:59.171962 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:59.139971 31204 raft_consensus.cc:2749] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:47:59.120118 31171 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:59.153579 31053 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:59.163096 31194 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:59.196592 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 52736.1us
W20260807 01:47:59.179533 31271 consensus_meta.cc:287] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade: Time spent flushing consensus metadata: real 0.816s user 0.009s sys 0.004s
I20260807 01:47:59.174276 31003 raft_consensus.cc:493] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:47:59.210417 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 95082.9us
W20260807 01:47:59.251941 27287 connection.cc:256] RPC call timeout handler was delayed by -1.3920281350146979s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15280000000000002s for a response
W20260807 01:47:59.255158 31444 log.cc:927] Time spent T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.126s user 0.001s sys 0.000s
I20260807 01:47:59.255806 31458 raft_consensus.cc:2749] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260807 01:47:59.257382 31445 log.cc:927] Time spent T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.131s user 0.000s sys 0.000s
I20260807 01:47:59.218439 31271 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:59.263408 31271 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:59.179764 31420 consensus_queue.cc:237] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:59.231225 31003 raft_consensus.cc:515] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:59.270203 31003 leader_election.cc:290] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:47:59.268028 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 134992us
W20260807 01:47:59.252393 27287 connection.cc:256] RPC call timeout handler was delayed by -1.36772813485004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1555s for a response
W20260807 01:47:59.277007 27287 connection.cc:256] RPC call timeout handler was delayed by -1.3650281350128353s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15580000000000002s for a response
W20260807 01:47:59.277340 27287 connection.cc:256] RPC call timeout handler was delayed by -1.0842281349468976s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18700000000000003s for a response
I20260807 01:47:59.278424 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:59.279183 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:47:59.279850 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:47:59.282294 27064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:59.263283 31171 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:59.295436 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:59.251819 30888 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:59.209340 31190 consensus_queue.cc:1048] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260807 01:47:59.243270 31281 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:47:59.298671 27337 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:47:59.298691 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:59.299080 27328 raft_consensus.cc:1275] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 01:47:59.299907 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:59.313047 27326 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:47:59.318086 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:59.319497 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:59.320232 27325 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:47:59.319566 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:59.322834 31281 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:47:59.324494 31453 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:59.282920 31298 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:59.325179 31453 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:59.325575 31298 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:59.326851 27333 raft_consensus.cc:2468] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
W20260807 01:47:59.329725 31455 log.cc:927] Time spent T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.099s user 0.000s sys 0.000s
W20260807 01:47:59.333361 31463 log.cc:927] Time spent T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.084s user 0.000s sys 0.001s
I20260807 01:47:59.335398 27229 raft_consensus.cc:1240] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 7. Current term is 8. Ops: []
I20260807 01:47:59.340420 27172 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
W20260807 01:47:59.301760 31443 log.cc:927] Time spent T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.068s user 0.003s sys 0.000s
I20260807 01:47:59.351982 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:59.347440 27337 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
I20260807 01:47:59.359861 28655 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:47:59.300743 31306 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:59.395190 27105 raft_consensus.cc:2468] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:59.405558 27108 raft_consensus.cc:1275] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260807 01:47:59.407589 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:59.410645 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:59.373217 31298 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:59.401678 31189 raft_consensus.cc:2804] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:47:59.373824 31466 raft_consensus.cc:2804] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:47:59.427795 30854 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:47:59.430469 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:59.456808 27337 raft_consensus.cc:2393] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 7: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:59.457116 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:59.457849 27104 raft_consensus.cc:2468] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:47:59.437456 31473 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:59.439522 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:59.371665 31270 consensus_queue.cc:1048] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
W20260807 01:47:59.477123 27170 leader_election.cc:336] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.936s (SENT)
I20260807 01:47:59.502023 27325 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8.
I20260807 01:47:59.502446 27170 leader_election.cc:304] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:59.430387 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:59.311357 30888 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:59.570756 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:47:59.459403 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 349852us
I20260807 01:47:59.474897 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:59.503624 27338 raft_consensus.cc:2393] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 3: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
W20260807 01:47:59.587949 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 110877us
W20260807 01:47:59.504391 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:47:59.504752 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 291337us
I20260807 01:47:59.421121 31329 raft_consensus.cc:697] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:59.450546 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:47:59.679708 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:59.648486 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:59.665057 27326 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 10.
I20260807 01:47:59.589336 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:47:59.741523 27170 leader_election.cc:336] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.959s (SENT)
I20260807 01:47:59.749375 27170 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:59.748404 27328 raft_consensus.cc:2468] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
W20260807 01:47:59.759517 28662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:47:59.764446 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:59.765625 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:59.765969 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:59.766256 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:59.766497 27333 raft_consensus.cc:2393] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 5: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:59.766573 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:59.767467 27327 raft_consensus.cc:2393] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 5: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:47:59.767311 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:59.768096 27328 raft_consensus.cc:2393] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 5: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:59.768705 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:59.673272 31269 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:47:59.673905 31426 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:59.504956 31466 raft_consensus.cc:697] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:47:59.722116 31479 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:47:59.660742 31480 raft_consensus.cc:2749] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:47:59.673493 31281 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:59.769909 27327 raft_consensus.cc:2393] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 4: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:47:59.775516 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:59.648070 31189 raft_consensus.cc:697] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 7 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:47:59.508097 31397 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:59.793015 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:47:59.762593 31203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.765s user 0.032s sys 0.054s
W20260807 01:47:59.780537 27170 leader_election.cc:336] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 2.027s (SENT)
I20260807 01:47:59.759316 31207 raft_consensus.cc:2749] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260807 01:47:59.769894 31420 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.502s user 0.007s sys 0.087s
I20260807 01:47:59.787003 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:59.793807 27105 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
I20260807 01:47:59.762087 31453 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:47:59.796373 27170 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:47:59.798162 28708 raft_consensus.cc:1275] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260807 01:47:59.756229 31171 consensus_queue.cc:1059] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260807 01:47:59.796048 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:59.587816 31473 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:59.501031 30854 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:47:59.804332 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:47:59.849593 27110 raft_consensus.cc:1240] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 3. Current term is 4. Ops: []
W20260807 01:47:59.917945 27331 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s user 0.011s sys 0.022s
I20260807 01:47:59.941059 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:47:59.791540 31053 consensus_meta.cc:287] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.601s user 0.007s sys 0.003s
I20260807 01:47:59.960812 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:59.984627 27103 raft_consensus.cc:3060] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:47:59.980278 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:59.979501 27336 raft_consensus.cc:2410] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260807 01:47:59.966822 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:47:59.875528 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:47:59.927045 28706 raft_consensus.cc:2368] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 6. Current term is 7.
W20260807 01:47:59.981927 27331 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s user 0.012s sys 0.024s
W20260807 01:47:59.990963 31494 log.cc:927] Time spent T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.051s user 0.000s sys 0.000s
I20260807 01:47:59.975211 26952 catalog_manager.cc:5697] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:59.992533 28713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48362 (request call id 773) took 1493 ms. Trace:
I20260807 01:48:00.012961 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:47:59.992172 28702 raft_consensus.cc:2955] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 5 to 7, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
W20260807 01:47:59.993731 31460 log.cc:927] Time spent T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.745s user 0.001s sys 0.001s
W20260807 01:47:59.995889 31150 consensus_meta.cc:287] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.937s user 0.011s sys 0.001s
I20260807 01:47:59.992172 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:00.009974 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:47:59.962280 31466 consensus_queue.cc:237] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
W20260807 01:47:59.970808 31471 log.cc:927] Time spent T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.199s user 0.000s sys 0.000s
I20260807 01:47:59.972971 31189 consensus_queue.cc:237] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:47:59.998865 31476 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s user 0.033s sys 0.079s
I20260807 01:48:00.042541 27101 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:47:59.941808 30888 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:47:59.785233 31426 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:00.059389 30854 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:00.060011 31500 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:00.060654 31500 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:00.061030 31353 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:00.042954 26952 catalog_manager.cc:5697] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:47:59.999236 27325 raft_consensus.cc:3060] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:47:59.669375 31204 raft_consensus.cc:2955] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 2 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
W20260807 01:48:00.046200 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 306011us
I20260807 01:48:00.046825 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:00.048174 27100 raft_consensus.cc:3060] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:47:59.873054 31281 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:47:59.804812 31203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.802s user 0.032s sys 0.055s
I20260807 01:47:59.965791 31194 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:00.048362 27287 leader_election.cc:304] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
W20260807 01:48:00.097704 27063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:00.053644 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:00.066192 31420 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:00.044078 28713 rpcz_store.cc:276] 0807 01:47:58.499341 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:58.551594 (+ 52253us) service_pool.cc:224] Handling call
0807 01:47:58.551974 (+ 380us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:59.158118 (+606144us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:47:59.992278 (+834160us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b5400699454. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:47:59.992469 (+ 191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1275337472}
I20260807 01:48:00.056577 31161 raft_consensus.cc:493] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:00.049314 31150 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:00.040944 31329 consensus_queue.cc:237] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:47:59.877414 31090 consensus_meta.cc:287] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.751s user 0.006s sys 0.005s
I20260807 01:48:00.044411 27333 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Advancing to term 11
I20260807 01:48:00.068382 27103 raft_consensus.cc:2468] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
I20260807 01:48:00.096216 31053 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:00.081856 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 307215us
I20260807 01:48:00.061374 31426 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:00.118894 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:00.058377 31511 raft_consensus.cc:2749] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 01:48:00.094559 27338 raft_consensus.cc:2468] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:48:00.096575 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:00.121999 27110 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:00.097128 27100 raft_consensus.cc:2468] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:47:59.800446 31479 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:00.097774 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:00.000607 27109 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
W20260807 01:48:00.100713 27063 connection.cc:256] RPC call timeout handler was delayed by -1.2074894739780575s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.192s for a response
I20260807 01:48:00.101032 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:00.100291 27336 raft_consensus.cc:2376] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8. Re-sending same reply.
W20260807 01:48:00.130059 27063 connection.cc:256] RPC call timeout handler was delayed by -1.1743332138285041s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1756s for a response
W20260807 01:48:00.130465 27063 connection.cc:256] RPC call timeout handler was delayed by -1.1687894738279283s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19630000000000003s for a response
W20260807 01:48:00.103463 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 55201.1us
I20260807 01:48:00.067320 31453 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:00.137035 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:00.113998 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
W20260807 01:48:00.052753 31476 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s user 0.037s sys 0.079s
I20260807 01:48:00.117972 31397 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:00.124094 28660 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:00.115365 31171 raft_consensus.cc:493] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:00.146066 31171 raft_consensus.cc:515] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:00.147392 31207 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:00.148092 31207 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:00.145608 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:00.151022 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:00.100499 31281 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:00.098364 31203 raft_consensus.cc:697] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 7 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:00.131421 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:48:00.130792 27063 connection.cc:256] RPC call timeout handler was delayed by -1.1516894737724215s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19820000000000002s for a response
W20260807 01:48:00.163810 27063 connection.cc:256] RPC call timeout handler was delayed by -1.0895894737914205s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20510000000000003s for a response
I20260807 01:48:00.163857 27338 raft_consensus.cc:2468] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
W20260807 01:48:00.164839 27063 leader_election.cc:336] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.619s (SENT)
I20260807 01:48:00.108825 30888 raft_consensus.cc:2749] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:48:00.136193 27328 raft_consensus.cc:2393] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 3: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:00.155033 27172 leader_election.cc:304] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:00.160079 27226 raft_consensus.cc:2955] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:48:00.123222 31409 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:00.160730 27287 leader_election.cc:304] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:00.176170 27213 raft_consensus.cc:1240] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 4. Current term is 5. Ops: []
I20260807 01:48:00.162323 27222 raft_consensus.cc:1275] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260807 01:48:00.166308 28660 leader_election.cc:304] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:48:00.169317 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:00.177731 27100 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 7
I20260807 01:48:00.170609 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:00.174281 27110 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
I20260807 01:48:00.174589 27172 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:00.108645 31420 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:00.148519 31171 leader_election.cc:290] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:00.176875 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:00.183024 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:00.183615 27109 raft_consensus.cc:1275] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 01:48:00.152768 31426 raft_consensus.cc:493] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:00.199334 27214 raft_consensus.cc:1275] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260807 01:48:00.145660 31485 raft_consensus.cc:3060] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:00.185025 27103 raft_consensus.cc:2468] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 2.
I20260807 01:48:00.120930 31090 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:00.180300 27336 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:48:00.182972 27229 raft_consensus.cc:3060] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:00.193602 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:00.123746 31053 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:00.194186 31192 raft_consensus.cc:2804] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader election won for term 5
W20260807 01:48:00.194868 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 57800.3us
I20260807 01:48:00.195307 31289 raft_consensus.cc:2749] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 01:48:00.175791 31207 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:00.197479 27063 leader_election.cc:336] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.660s (SENT)
I20260807 01:48:00.181975 31511 raft_consensus.cc:2804] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:48:00.153640 31453 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:00.213070 27063 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:00.198124 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:00.198331 31389 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:00.129316 27325 raft_consensus.cc:2468] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:48:00.113416 31161 raft_consensus.cc:515] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:00.204331 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:00.196058 31203 consensus_queue.cc:237] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:00.206957 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:00.207741 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:00.227833 27338 raft_consensus.cc:2376] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8. Re-sending same reply.
I20260807 01:48:00.227039 27099 raft_consensus.cc:1240] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 6. Current term is 7. Ops: []
I20260807 01:48:00.227039 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:00.228817 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:00.209522 27100 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
I20260807 01:48:00.229283 27336 raft_consensus.cc:2468] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:48:00.221197 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:00.221966 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:00.230707 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:00.231747 27325 raft_consensus.cc:3060] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:00.180619 31179 consensus_queue.cc:1059] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
W20260807 01:48:00.233090 31512 log.cc:927] Time spent T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.091s user 0.000s sys 0.000s
I20260807 01:48:00.235788 31090 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:00.181759 31480 consensus_queue.cc:1048] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260807 01:48:00.237214 31281 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:00.238930 31420 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:00.239491 31187 raft_consensus.cc:993] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:48:00.231585 27230 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
I20260807 01:48:00.231623 27214 raft_consensus.cc:2468] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:48:00.245365 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:48:00.246073 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 48643.6us
W20260807 01:48:00.246985 27063 connection.cc:256] RPC call timeout handler was delayed by -1.0625894737895578s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2081s for a response
I20260807 01:48:00.245576 27338 raft_consensus.cc:1275] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260807 01:48:00.249612 27063 connection.cc:256] RPC call timeout handler was delayed by -1.034668882843107s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2175s for a response
W20260807 01:48:00.250283 27063 leader_election.cc:336] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 2.152s (SENT)
I20260807 01:48:00.250715 27063 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:00.246609 27222 raft_consensus.cc:2468] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
W20260807 01:48:00.260732 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:00.262343 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
W20260807 01:48:00.263726 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 89205.2us
I20260807 01:48:00.184700 31409 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:00.269752 27336 raft_consensus.cc:1275] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260807 01:48:00.256330 27064 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:00.207043 31426 raft_consensus.cc:515] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:00.206176 31500 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:00.228374 31192 raft_consensus.cc:697] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:00.267426 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:00.228636 31416 raft_consensus.cc:2804] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:48:00.240295 31511 raft_consensus.cc:697] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:00.276153 27325 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:48:00.276396 27287 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:00.255148 31473 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:00.279877 27064 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:00.268961 31213 raft_consensus.cc:2749] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260807 01:48:00.280462 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 114270us
I20260807 01:48:00.285246 31192 consensus_queue.cc:237] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:00.306608 27229 raft_consensus.cc:2468] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
W20260807 01:48:00.306556 27064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:00.285777 27214 raft_consensus.cc:1275] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 01:48:00.286201 31289 raft_consensus.cc:2749] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 01:48:00.286765 31161 leader_election.cc:290] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:00.277335 31416 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:00.308725 27104 raft_consensus.cc:1275] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 01:48:00.230952 31389 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:00.295979 27333 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 11.
I20260807 01:48:00.316336 27216 raft_consensus.cc:1240] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3. Ops: []
W20260807 01:48:00.313138 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 72866.2us
I20260807 01:48:00.280750 31476 raft_consensus.cc:2749] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:48:00.290306 31194 raft_consensus.cc:2804] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:48:00.305866 31150 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:00.326248 27214 raft_consensus.cc:1275] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 01:48:00.324440 27287 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:00.320307 31416 raft_consensus.cc:3060] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:00.321624 31473 raft_consensus.cc:493] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:00.323572 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 73328.6us
W20260807 01:48:00.335767 27063 connection.cc:256] RPC call timeout handler was delayed by -1.0049894738476723s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21450000000000002s for a response
I20260807 01:48:00.308174 31509 raft_consensus.cc:2804] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:48:00.337577 31509 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:00.338129 31509 raft_consensus.cc:3060] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:00.318070 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:00.328997 28660 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:00.327368 31194 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:00.346072 27229 raft_consensus.cc:2468] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
I20260807 01:48:00.330407 27226 raft_consensus.cc:1240] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 4. Current term is 5. Ops: []
I20260807 01:48:00.352965 31518 raft_consensus.cc:2804] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:48:00.322188 31161 consensus_queue.cc:1048] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260807 01:48:00.359313 27287 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:00.358536 31194 raft_consensus.cc:3060] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:00.382086 28660 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:00.321579 31515 consensus_queue.cc:1048] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260807 01:48:00.388263 28660 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:00.380677 31397 consensus_queue.cc:1048] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.010s
I20260807 01:48:00.342446 31289 raft_consensus.cc:2749] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260807 01:48:00.396149 28715 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48350 (request call id 817) took 2088 ms (client timeout 2547 ms). Trace:
W20260807 01:48:00.399140 27170 leader_election.cc:336] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 2.547s (SENT)
W20260807 01:48:00.396580 28715 rpcz_store.cc:269] 0807 01:47:58.307821 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:58.307953 (+ 132us) service_pool.cc:224] Handling call
0807 01:48:00.395921 (+2087968us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400566dd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:00.396084 (+ 163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2597874944}
I20260807 01:48:00.346742 31476 raft_consensus.cc:493] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:00.335793 31189 consensus_queue.cc:1048] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
I20260807 01:48:00.405798 27170 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:00.358794 31479 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:00.386919 31053 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:00.411270 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 52087.9us
I20260807 01:48:00.395509 31396 raft_consensus.cc:2749] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 01:48:00.393580 31281 raft_consensus.cc:2749] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260807 01:48:00.402557 31518 raft_consensus.cc:697] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:00.418428 27287 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:00.394851 31454 raft_consensus.cc:2804] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:00.341161 31473 raft_consensus.cc:515] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:00.425928 27287 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
W20260807 01:48:00.432775 27063 leader_election.cc:336] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.575s (SENT)
I20260807 01:48:00.433197 27063 leader_election.cc:304] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:00.418143 31453 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:00.420490 31479 raft_consensus.cc:2749] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:48:00.419323 31530 raft_consensus.cc:2749] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:48:00.422144 31454 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:00.438344 31281 raft_consensus.cc:2764] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260807 01:48:00.442555 28706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52322 (request call id 669) took 1040 ms. Trace:
I20260807 01:48:00.442989 28706 rpcz_store.cc:276] 0807 01:47:59.402501 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:59.405229 (+ 2728us) service_pool.cc:224] Handling call
0807 01:48:00.442275 (+1037046us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b540070e254. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:00.442459 (+ 184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":290260736}
I20260807 01:48:00.338140 31511 consensus_queue.cc:237] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:00.446339 27218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57654 (request call id 1861) took 1519 ms. Trace:
I20260807 01:48:00.446769 27218 rpcz_store.cc:276] 0807 01:47:58.926901 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:58.930520 (+ 3619us) service_pool.cc:224] Handling call
0807 01:47:59.141106 (+210586us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:47:59.150621 (+ 9515us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:00.446092 (+1295471us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b5400125a50. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541dc8d 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:00.446269 (+ 177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":483428096}
I20260807 01:48:00.414345 31179 consensus_queue.cc:1048] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
W20260807 01:48:00.440383 27063 leader_election.cc:336] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.756s (SENT)
I20260807 01:48:00.449184 27063 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:00.451409 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:00.441236 31454 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:00.456714 31485 raft_consensus.cc:2749] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:48:00.434579 31426 leader_election.cc:290] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:00.440384 31213 raft_consensus.cc:2749] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:48:00.466023 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:00.466851 27218 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
W20260807 01:48:00.474681 27063 leader_election.cc:336] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.920s (SENT)
I20260807 01:48:00.475301 27063 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:00.478524 31426 raft_consensus.cc:2749] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260807 01:48:00.480235 31518 consensus_queue.cc:237] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:00.483691 31003 raft_consensus.cc:1064] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
W20260807 01:48:00.484202 31003 raft_consensus.cc:1068] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade: Unable to promote non-voter 57bb4985157447f0a3ab680fa0026f12: Illegal state: Leader has not yet committed an operation in its own term
I20260807 01:48:00.480377 28710 raft_consensus.cc:3055] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:48:00.488529 28710 raft_consensus.cc:740] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:00.489475 28710 consensus_queue.cc:260] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:00.491056 28710 raft_consensus.cc:3060] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:00.409708 31476 raft_consensus.cc:515] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:00.500041 31476 leader_election.cc:290] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:00.501183 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 102081us
W20260807 01:48:00.500166 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:00.503809 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 77974us
W20260807 01:48:00.521445 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 45927.6us
I20260807 01:48:00.493932 31509 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:00.488420 31389 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:48:00.527355 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 52722.9us
I20260807 01:48:00.548262 27218 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
W20260807 01:48:00.549238 27170 leader_election.cc:336] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 3.028s (SENT)
I20260807 01:48:00.549636 27170 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
W20260807 01:48:00.541620 27063 leader_election.cc:336] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.963s (SENT)
I20260807 01:48:00.561837 31479 consensus_queue.cc:1059] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:48:00.562862 31479 consensus_queue.cc:1243] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer 3262a9d183914b699227fdba0ca5bed3 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260807 01:48:00.563488 31003 raft_consensus.cc:3055] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:48:00.563894 31003 raft_consensus.cc:740] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:00.564574 31194 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:00.565235 31003 consensus_queue.cc:260] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:00.566207 27063 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:00.569411 31454 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:00.570981 27102 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:41078 (request call id 85) took 1428 ms. Trace:
I20260807 01:48:00.571991 31187 raft_consensus.cc:1081] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Signalling peer 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:48:00.582207 28710 raft_consensus.cc:1275] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260807 01:48:00.588109 28710 pending_rounds.cc:85] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12: Aborting all ops after (but not including) 5
I20260807 01:48:00.588425 28710 pending_rounds.cc:107] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260807 01:48:00.588881 28710 raft_consensus.cc:2889] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 01:48:00.588485 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:48:00.589793 31484 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260807 01:48:00.590128 27337 raft_consensus.cc:2376] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3. Re-sending same reply.
I20260807 01:48:00.592442 27103 raft_consensus.cc:1275] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 01:48:00.572233 31416 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:00.593027 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:00.593698 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:00.594482 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:00.581591 27102 rpcz_store.cc:276] 0807 01:47:59.142980 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:59.143104 (+ 124us) service_pool.cc:224] Handling call
0807 01:48:00.570590 (+1427486us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b54004752a0. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5539b900 00007f7d55418a32 00007f7d5a41841a 00007f7d5396c934 00007f7d5396c8c2 00007f7d5396c7f1 00007f7d5396c76c 00007f7d5396ba22 00007f7d5375807c 00007f7d537574b6
0807 01:48:00.570875 (+ 285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":785896704}
I20260807 01:48:00.594821 27100 raft_consensus.cc:2393] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 4: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:00.596618 31416 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:00.598217 27101 raft_consensus.cc:1275] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 11 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260807 01:48:00.592271 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:00.599665 27101 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:41078
I20260807 01:48:00.601409 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:00.601699 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:00.602309 27102 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:00.603359 31454 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:00.547703 31476 raft_consensus.cc:2749] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 01:48:00.607473 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:00.608129 27110 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:48:00.606855 31281 consensus_queue.cc:1059] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260807 01:48:00.609262 31473 leader_election.cc:290] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:00.605918 31458 raft_consensus.cc:2749] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:00.574437 31509 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:00.613847 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:00.617352 27102 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
W20260807 01:48:00.614415 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 72812.8us
W20260807 01:48:00.621170 27063 leader_election.cc:336] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.982s (SENT)
I20260807 01:48:00.574342 31003 raft_consensus.cc:3060] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:00.623066 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:00.623800 27102 raft_consensus.cc:3060] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:00.614959 30888 consensus_queue.cc:1059] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260807 01:48:00.625123 31458 consensus_queue.cc:1048] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 01:48:00.625860 31194 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:00.627152 27172 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:00.627995 31458 raft_consensus.cc:2804] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:00.628381 31458 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:00.628679 31458 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:00.621567 31503 raft_consensus.cc:2749] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 01:48:00.633771 27172 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
W20260807 01:48:00.634656 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 85450.3us
I20260807 01:48:00.636742 31530 raft_consensus.cc:2804] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:48:00.637389 31530 raft_consensus.cc:697] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 7 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:00.637981 30854 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:00.638542 30854 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:00.638373 31530 consensus_queue.cc:237] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:00.640739 30854 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:00.655827 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:00.656546 27103 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:48:00.649847 31409 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:00.662015 27063 leader_election.cc:336] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.987s (SENT)
I20260807 01:48:00.612958 31289 consensus_queue.cc:1059] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
W20260807 01:48:00.667451 27063 leader_election.cc:336] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 2.051s (SENT)
I20260807 01:48:00.667956 27063 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:00.668468 27172 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:00.642896 28707 raft_consensus.cc:2955] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:48:00.635828 31511 consensus_queue.cc:1048] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260807 01:48:00.661576 31537 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:00.675127 31474 raft_consensus.cc:2749] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:48:00.672614 31289 consensus_queue.cc:1243] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer 3262a9d183914b699227fdba0ca5bed3 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260807 01:48:00.677289 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:00.677836 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:00.679843 27063 leader_election.cc:336] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 2.081s (SENT)
I20260807 01:48:00.673048 31466 raft_consensus.cc:2804] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:00.678767 31289 consensus_queue.cc:1059] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260807 01:48:00.681361 31289 consensus_queue.cc:1243] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer 3262a9d183914b699227fdba0ca5bed3 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260807 01:48:00.679814 27102 raft_consensus.cc:2468] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:48:00.682760 27063 leader_election.cc:304] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:00.679253 31537 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:00.684546 27326 raft_consensus.cc:1240] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 4. Current term is 5. Ops: []
I20260807 01:48:00.686188 31537 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:00.683521 31466 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:00.687485 31466 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:00.689946 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:00.690790 27325 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:00.694919 27172 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
W20260807 01:48:00.693732 27170 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 3.020s (SENT) [suppressed 59 similar messages]
I20260807 01:48:00.716707 27216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57654 (request call id 1867) took 1619 ms. Trace:
I20260807 01:48:00.722389 27325 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
I20260807 01:48:00.717206 27216 rpcz_store.cc:276] 0807 01:47:59.096606 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:59.137608 (+ 41002us) service_pool.cc:224] Handling call
0807 01:47:59.137947 (+ 339us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:00.321018 (+1183071us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:00.715371 (+394353us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540042bb54. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:00.715554 (+ 183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2709616512}
W20260807 01:48:00.725183 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 332482us
I20260807 01:48:00.729688 27222 raft_consensus.cc:1240] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260807 01:48:00.726684 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:00.731907 27067 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:00.710955 31453 raft_consensus.cc:2804] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:48:00.737047 27067 leader_election.cc:400] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225) with higher term. Message: Invalid argument: T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 6. Current term is 7.
I20260807 01:48:00.737154 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:00.739114 27067 leader_election.cc:403] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260807 01:48:00.723219 31103 raft_consensus.cc:2749] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260807 01:48:00.737609 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 57805us
W20260807 01:48:00.747447 27063 leader_election.cc:336] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 2.145s (SENT)
I20260807 01:48:00.749388 27063 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:00.752657 28660 leader_election.cc:304] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:00.738857 31474 raft_consensus.cc:2749] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:00.758786 27175 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:00.761301 28660 leader_election.cc:304] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:00.750262 31453 raft_consensus.cc:697] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:00.872550 31385 raft_consensus.cc:2749] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260807 01:48:00.874949 30742 consensus_queue.cc:1059] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260807 01:48:00.876298 31485 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:00.876828 31485 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:00.872695 31477 raft_consensus.cc:2749] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:00.879314 31526 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:48:00.883771 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:00.886884 27333 raft_consensus.cc:2468] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:48:00.889325 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:00.890008 27099 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:00.893283 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:00.743400 31509 raft_consensus.cc:2749] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225) with higher term. Message: Invalid argument: T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 6. Current term is 7.
I20260807 01:48:00.907548 27333 raft_consensus.cc:1275] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260807 01:48:00.866961 31270 raft_consensus.cc:2804] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260807 01:48:00.895392 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 147974us
W20260807 01:48:00.898545 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 137314us
I20260807 01:48:00.899681 27325 raft_consensus.cc:1275] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 01:48:00.911439 28660 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:00.914521 27099 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:48:00.905311 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:00.963572 28710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:34614
I20260807 01:48:00.964315 28710 raft_consensus.cc:606] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:48:00.922575 27327 raft_consensus.cc:2468] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
W20260807 01:48:00.966836 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 53780.5us
W20260807 01:48:00.968806 27063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:00.973742 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:00.907051 31122 raft_consensus.cc:2804] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:48:00.890081 31161 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:00.977232 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 65901.3us
I20260807 01:48:00.899117 28702 raft_consensus.cc:1240] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5. Ops: []
I20260807 01:48:00.979415 28709 raft_consensus.cc:1275] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 01:48:00.863070 31370 consensus_queue.cc:1059] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.102s
I20260807 01:48:00.982470 31122 raft_consensus.cc:493] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:00.905373 31190 raft_consensus.cc:3055] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 01:48:00.987306 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:00.879804 31416 consensus_queue.cc:1048] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.092s
I20260807 01:48:00.988128 28709 raft_consensus.cc:2393] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 6: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:00.972323 31270 raft_consensus.cc:493] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:00.983270 31122 raft_consensus.cc:3060] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:00.979645 30742 raft_consensus.cc:493] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:00.976946 31203 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:00.998447 31270 raft_consensus.cc:3060] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:01.012183 31171 raft_consensus.cc:3055] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:48:00.986413 31190 raft_consensus.cc:740] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:00.973053 31269 consensus_queue.cc:1048] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260807 01:48:00.996587 31397 raft_consensus.cc:2804] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:48:00.900889 31526 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:00.999408 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:01.012743 31171 raft_consensus.cc:740] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:01.028266 31466 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:00.986610 31161 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:01.054123 27101 raft_consensus.cc:606] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Received request to transfer leadership
I20260807 01:48:01.010794 30742 raft_consensus.cc:515] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:00.997599 31477 consensus_queue.cc:1048] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260807 01:48:01.067579 31466 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:01.073604 28715 raft_consensus.cc:2393] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 4: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:01.074766 30742 leader_election.cc:290] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:01.068778 31465 consensus_queue.cc:1059] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260807 01:48:01.043535 31397 raft_consensus.cc:697] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 7 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:01.016531 31203 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:01.013468 31453 consensus_queue.cc:237] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:01.081470 31161 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:48:01.112133 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:01.132085 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 509975us
I20260807 01:48:01.153117 31397 consensus_queue.cc:237] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:01.164348 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:01.165133 27227 raft_consensus.cc:2468] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:48:01.167965 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:48:01.163007 31458 consensus_meta.cc:287] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.534s user 0.015s sys 0.000s
I20260807 01:48:01.185144 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:01.195523 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:01.198050 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:01.198890 27213 raft_consensus.cc:2468] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:48:01.167104 31485 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:01.218902 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:01.192863 27227 raft_consensus.cc:2955] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 5 to 7, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:48:01.225286 27064 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:01.198783 31122 raft_consensus.cc:515] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:01.234781 31526 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:01.236383 26952 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46138 (request call id 398) took 3075 ms. Trace:
I20260807 01:48:01.231197 31503 raft_consensus.cc:493] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:01.240396 31503 raft_consensus.cc:515] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:01.244840 28715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
mode: GRACEFUL
new_leader_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:34614
I20260807 01:48:01.245500 28715 raft_consensus.cc:606] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 7 LEADER]: Received request to transfer leadership to TS 3478729505b046c2a0051c2d0130873f
I20260807 01:48:01.246932 31289 raft_consensus.cc:3055] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 7 LEADER]: Stepping down as leader of term 7
I20260807 01:48:01.242897 31122 leader_election.cc:290] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:01.247671 31289 raft_consensus.cc:740] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:01.252049 31289 consensus_queue.cc:260] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:01.253425 31289 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:01.253883 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:01.252286 31473 raft_consensus.cc:2955] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Committing config change with OpId 6.7: config changed from index 5 to 7, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:48:01.256919 28702 raft_consensus.cc:1275] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260807 01:48:01.254683 31409 raft_consensus.cc:2804] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:48:01.259673 31409 raft_consensus.cc:697] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 7 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:01.259866 28711 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:34614
I20260807 01:48:01.260591 28711 raft_consensus.cc:606] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:48:01.260954 31409 consensus_queue.cc:237] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:01.263343 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:01.193159 31458 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:01.236801 26952 rpcz_store.cc:276] 0807 01:47:58.160803 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:47:58.175964 (+ 15161us) service_pool.cc:224] Handling call
0807 01:48:01.236270 (+3060306us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 01:48:00.223199 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 01:48:00.223809 (+ 610us) write_op.cc:432] Acquiring schema lock in shared mode
0807 01:48:00.223849 (+ 40us) write_op.cc:435] Acquired schema lock
0807 01:48:00.223867 (+ 18us) tablet.cc:662] Decoding operations
0807 01:48:00.224289 (+ 422us) write_op.cc:620] Acquiring the partition lock for write op
0807 01:48:00.224376 (+ 87us) write_op.cc:641] Partition lock acquired for write op
0807 01:48:00.224405 (+ 29us) tablet.cc:685] Acquiring locks for 2 operations
0807 01:48:00.224784 (+ 379us) tablet.cc:701] Row locks acquired
0807 01:48:00.224799 (+ 15us) write_op.cc:260] PREPARE: finished
0807 01:48:00.224976 (+ 177us) write_op.cc:270] Start()
0807 01:48:00.225489 (+ 513us) write_op.cc:276] Timestamp: P: 1786067280224916 usec, L: 0
0807 01:48:00.225531 (+ 42us) op_driver.cc:348] REPLICATION: starting
0807 01:48:00.268410 (+ 42879us) log.cc:844] Serialized 911 byte log entry
0807 01:48:00.490041 (+221631us) spinlock_profiling.cc:243] Waited 47.1 ms on lock 0x7b54002afe20. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55388470 00007f7d55395a98 00007f7d553956da 00007f7d55419d45 00007f7d5541a1e9 00007f7d58e73f61 00007f7d58e73738 00007f7d58e783d1 00007f7d58e78389 00007f7d58e78319 00007f7d58e782e1
0807 01:48:00.522306 (+ 32265us) op_driver.cc:464] REPLICATION: finished
0807 01:48:00.539254 (+ 16948us) write_op.cc:301] APPLY: starting
0807 01:48:00.540991 (+ 1737us) tablet.cc:1367] starting BulkCheckPresence
0807 01:48:00.582211 (+ 41220us) tablet.cc:1370] finished BulkCheckPresence
0807 01:48:00.587884 (+ 5673us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 01:48:00.650474 (+ 62590us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 01:48:00.650689 (+ 215us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0807 01:48:00.650735 (+ 46us) write_op.cc:312] APPLY: finished
0807 01:48:00.650938 (+ 203us) log.cc:844] Serialized 29 byte log entry
0807 01:48:00.651117 (+ 179us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:48:00.651206 (+ 89us) write_op.cc:454] Released schema lock
0807 01:48:00.651255 (+ 49us) write_op.cc:341] FINISH: Updating metrics
0807 01:48:00.651702 (+ 447us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:48:00.651730 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3787,"reported_tablets":15,"child_traces":[["op",{"apply.queue_time_us":13770,"apply.run_cpu_time_us":2527,"apply.run_wall_time_us":115288,"num_ops":2,"prepare.queue_time_us":123146,"prepare.run_cpu_time_us":8053,"prepare.run_wall_time_us":267187,"raft.queue_time_us":140447,"raft.run_cpu_time_us":10081,"raft.run_wall_time_us":17271,"replication_time_us":296555,"spinlock_wait_cycles":108254592,"thread_start_us":272925,"threads_started":4,"wal-append.queue_time_us":53853}]]}
I20260807 01:48:01.288347 28715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:34614
I20260807 01:48:01.289287 28715 raft_consensus.cc:606] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:48:01.289737 28715 raft_consensus.cc:614] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 01:48:01.259673 31203 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:01.264272 31503 leader_election.cc:290] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:01.306078 24592 auto_leader_rebalancer.cc:467] leader transfer for tablet f27eab393b1f4f21aad32707749effa3 (from 57bb4985157447f0a3ab680fa0026f12 to 3262a9d183914b699227fdba0ca5bed3) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 01:48:01.307145 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260807 01:48:01.314615 27175 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:01.318495 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:48:01.319711 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:48:01.322656 27175 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
W20260807 01:48:01.312302 31530 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s user 0.010s sys 0.023s
W20260807 01:48:01.309044 30888 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.637s user 0.019s sys 0.023s
I20260807 01:48:01.346285 27064 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
W20260807 01:48:01.336191 31479 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.680s user 0.063s sys 0.096s
I20260807 01:48:01.334898 31416 raft_consensus.cc:2749] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260807 01:48:01.347728 30888 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.681s user 0.020s sys 0.024s
I20260807 01:48:01.342829 31203 raft_consensus.cc:2749] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260807 01:48:01.333695 31530 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s user 0.010s sys 0.024s
I20260807 01:48:01.385340 27064 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:01.351356 31467 raft_consensus.cc:2804] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:01.399196 31467 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:01.399892 31467 raft_consensus.cc:3060] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:01.344781 31190 consensus_queue.cc:260] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:01.398293 31003 raft_consensus.cc:3055] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Stepping down as leader of term 6
I20260807 01:48:01.349431 31477 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:48:01.408777 27063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:01.380002 31479 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.727s user 0.064s sys 0.097s
I20260807 01:48:01.421562 31003 raft_consensus.cc:740] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:01.421307 31203 consensus_queue.cc:1048] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260807 01:48:01.422097 30888 raft_consensus.cc:3055] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Stepping down as leader of term 4
W20260807 01:48:01.524230 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 138942us
I20260807 01:48:01.460322 31171 consensus_queue.cc:260] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:01.521452 31270 consensus_meta.cc:287] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 0.502s user 0.015s sys 0.003s
I20260807 01:48:01.553440 31503 raft_consensus.cc:2804] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:48:01.567224 31503 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:01.516433 30888 raft_consensus.cc:740] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:01.555335 31509 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:01.552698 31477 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:01.582561 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:01.560703 31281 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.899s user 0.006s sys 0.029s
W20260807 01:48:01.658114 27289 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.914s user 0.032s sys 0.031s
W20260807 01:48:01.659018 27289 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.916s user 0.032s sys 0.032s
W20260807 01:48:01.659567 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 916769us
I20260807 01:48:01.563925 31458 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:01.569561 31190 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Advancing to term 7
W20260807 01:48:01.686214 27108 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.672s user 0.020s sys 0.008s
W20260807 01:48:01.668889 27337 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.986s user 0.029s sys 0.059s
W20260807 01:48:01.690032 27337 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.008s user 0.030s sys 0.059s
I20260807 01:48:01.611321 31171 raft_consensus.cc:3060] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:01.652273 31509 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:01.680754 27172 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.703s user 0.040s sys 0.047s
W20260807 01:48:01.715516 27172 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.738s user 0.041s sys 0.047s
I20260807 01:48:01.634407 31503 raft_consensus.cc:3060] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
W20260807 01:48:01.689427 28661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:01.566382 31270 raft_consensus.cc:515] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:01.692945 28655 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.726s user 0.011s sys 0.045s
I20260807 01:48:01.659106 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
W20260807 01:48:01.717334 28661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:01.717298 28655 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.750s user 0.011s sys 0.045s
W20260807 01:48:01.718513 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 820014us
W20260807 01:48:01.666388 27229 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.918s user 0.008s sys 0.026s
W20260807 01:48:01.719559 27229 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.971s user 0.009s sys 0.026s
I20260807 01:48:01.569351 31003 consensus_queue.cc:260] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:01.656374 31281 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.997s user 0.007s sys 0.029s
W20260807 01:48:01.746088 28660 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.768s user 0.017s sys 0.034s
W20260807 01:48:01.747016 28660 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.769s user 0.017s sys 0.034s
W20260807 01:48:01.747295 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 769308us
W20260807 01:48:01.721709 27212 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.665s user 0.000s sys 0.012s
W20260807 01:48:01.719607 27108 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.710s user 0.021s sys 0.008s
W20260807 01:48:01.748685 27212 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s user 0.000s sys 0.012s
W20260807 01:48:01.758801 28709 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.746s user 0.003s sys 0.018s
W20260807 01:48:01.759349 28709 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.746s user 0.003s sys 0.018s
W20260807 01:48:01.749541 28706 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.638s user 0.003s sys 0.025s
W20260807 01:48:01.704411 27110 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.800s user 0.000s sys 0.030s
I20260807 01:48:01.748072 31281 consensus_queue.cc:1243] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer 3262a9d183914b699227fdba0ca5bed3 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260807 01:48:01.740377 28707 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.858s user 0.017s sys 0.010s
I20260807 01:48:01.751348 31458 raft_consensus.cc:493] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:01.768972 27110 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.864s user 0.001s sys 0.030s
I20260807 01:48:01.761010 28702 raft_consensus.cc:2393] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 6: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
W20260807 01:48:01.794965 27286 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s user 0.013s sys 0.027s
W20260807 01:48:01.797453 27286 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.651s user 0.013s sys 0.027s
I20260807 01:48:01.767547 31003 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:01.784299 31117 consensus_queue.cc:1048] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260807 01:48:01.796504 28714 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.714s user 0.022s sys 0.040s
W20260807 01:48:01.771160 28706 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s user 0.003s sys 0.025s
I20260807 01:48:01.744750 31479 consensus_queue.cc:1048] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
W20260807 01:48:01.773219 27109 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.697s user 0.008s sys 0.009s
W20260807 01:48:01.802498 27109 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.729s user 0.009s sys 0.009s
I20260807 01:48:01.803519 27109 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 2994) took 1182 ms. Trace:
W20260807 01:48:01.706941 31490 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.684s user 0.020s sys 0.040s
W20260807 01:48:01.763185 31396 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.790s user 0.011s sys 0.050s
I20260807 01:48:01.785306 31122 consensus_queue.cc:1048] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:48:01.741519 30888 consensus_queue.cc:260] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:01.800979 28714 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.719s user 0.023s sys 0.041s
W20260807 01:48:01.773926 28707 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.892s user 0.018s sys 0.010s
W20260807 01:48:01.776721 31187 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s user 0.000s sys 0.013s
W20260807 01:48:01.805243 31490 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.792s user 0.021s sys 0.040s
I20260807 01:48:01.965724 27101 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:41078 (request call id 86) took 1366 ms. Trace:
I20260807 01:48:01.803908 27109 rpcz_store.cc:276] 0807 01:48:00.620813 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:00.620957 (+ 144us) service_pool.cc:224] Handling call
0807 01:48:00.630110 (+ 9153us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:00.630344 (+ 234us) raft_consensus.cc:1534] Early marking committed up to index 7
0807 01:48:01.073161 (+442817us) log.cc:844] Serialized 14 byte log entry
0807 01:48:01.802954 (+729793us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:01.803011 (+ 57us) raft_consensus.cc:1645] Marking committed up to 7
0807 01:48:01.803053 (+ 42us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:01.803323 (+ 270us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:01.803452 (+ 129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4382,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":820,"thread_start_us":1171452,"threads_started":2}
W20260807 01:48:01.807514 31396 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.834s user 0.011s sys 0.050s
I20260807 01:48:01.790931 31458 raft_consensus.cc:515] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:01.807675 31397 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s user 0.004s sys 0.016s
I20260807 01:48:01.791992 31485 consensus_queue.cc:1048] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
W20260807 01:48:01.807493 27287 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s user 0.000s sys 0.025s
W20260807 01:48:01.807775 28710 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.620s user 0.010s sys 0.009s
I20260807 01:48:01.911906 31477 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:01.966392 31187 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.701s user 0.000s sys 0.014s
I20260807 01:48:02.040701 27108 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 2999) took 1142 ms. Trace:
I20260807 01:48:01.965005 31203 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:48:01.964815 31453 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s user 0.021s sys 0.025s
W20260807 01:48:01.979151 31397 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.696s user 0.005s sys 0.017s
I20260807 01:48:01.966187 27101 rpcz_store.cc:276] 0807 01:48:00.599392 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:00.599548 (+ 156us) service_pool.cc:224] Handling call
0807 01:48:01.965450 (+1365902us) spinlock_profiling.cc:243] Waited 454 ms on lock 0x7b54003183d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d55418c33 00007f7d5a41841a 00007f7d5396c934 00007f7d5396c8c2 00007f7d5396c7f1 00007f7d5396c76c 00007f7d5396ba22 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:01.965644 (+ 194us) inbound_call.cc:177] Queueing failure response
Metrics: {"spinlock_wait_cycles":1043672960}
W20260807 01:48:02.043512 31561 log.cc:927] Time spent T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.297s user 0.000s sys 0.001s
I20260807 01:48:02.079620 31203 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:02.051105 28710 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.864s user 0.011s sys 0.009s
I20260807 01:48:02.041191 31509 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:01.804975 31500 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.804s user 0.033s sys 0.047s
W20260807 01:48:02.063176 31567 log.cc:927] Time spent T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.086s user 0.000s sys 0.000s
W20260807 01:48:02.051030 27287 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.892s user 0.002s sys 0.033s
W20260807 01:48:02.111760 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 953274us
I20260807 01:48:02.113716 27287 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:02.043157 31568 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:02.086848 27104 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 2988) took 1486 ms. Trace:
I20260807 01:48:02.080742 31117 raft_consensus.cc:993] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12: : Instructing follower 3262a9d183914b699227fdba0ca5bed3 to start an election
W20260807 01:48:02.080247 31453 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.791s user 0.021s sys 0.030s
I20260807 01:48:02.073030 27108 rpcz_store.cc:276] 0807 01:48:00.897866 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:00.898282 (+ 416us) service_pool.cc:224] Handling call
0807 01:48:00.905668 (+ 7386us) raft_consensus.cc:1471] Updating replica for 2 ops
0807 01:48:00.906076 (+ 408us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:00.906105 (+ 29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0807 01:48:00.985020 (+ 78915us) log.cc:844] Serialized 52 byte log entry
0807 01:48:01.752035 (+767015us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:48:01.752107 (+ 72us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:01.752236 (+ 129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:02.040420 (+288184us) raft_consensus.cc:1685] finished
0807 01:48:02.040443 (+ 23us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:02.040629 (+ 186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1239,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":17759,"thread_start_us":739238,"threads_started":1,"wal-append.queue_time_us":809599}
I20260807 01:48:02.123747 27104 rpcz_store.cc:276] 0807 01:48:00.600710 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:00.602019 (+ 1309us) service_pool.cc:224] Handling call
0807 01:48:00.602515 (+ 496us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:00.613433 (+ 10918us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:00.613471 (+ 38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:00.614008 (+ 537us) log.cc:844] Serialized 26 byte log entry
0807 01:48:01.116264 (+502256us) raft_consensus.cc:1645] Marking committed up to 5
0807 01:48:01.116313 (+ 49us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:01.116395 (+ 82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:02.086641 (+970246us) raft_consensus.cc:1685] finished
0807 01:48:02.086662 (+ 21us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:02.086787 (+ 125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34457,"raft.queue_time_us":11340,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":16979,"thread_start_us":457438,"threads_started":1,"wal-append.queue_time_us":1046154}
W20260807 01:48:02.119132 31500 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.119s user 0.035s sys 0.047s
I20260807 01:48:02.086794 31458 leader_election.cc:290] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:02.087330 28707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52322 (request call id 674) took 2090 ms. Trace:
W20260807 01:48:02.123374 31467 consensus_meta.cc:287] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.723s user 0.017s sys 0.006s
W20260807 01:48:02.137447 27064 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s user 0.012s sys 0.014s
W20260807 01:48:02.156023 27064 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s user 0.012s sys 0.014s
W20260807 01:48:02.156955 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 599397us
W20260807 01:48:02.146699 27230 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.745s user 0.010s sys 0.009s
W20260807 01:48:02.159090 28661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:02.198685 27230 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.815s user 0.010s sys 0.010s
W20260807 01:48:02.226900 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:02.132772 31397 raft_consensus.cc:1081] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Signalling peer 3262a9d183914b699227fdba0ca5bed3 to start an election
W20260807 01:48:02.136217 31290 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.914s user 0.009s sys 0.072s
I20260807 01:48:02.126704 31281 raft_consensus.cc:2804] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:02.133452 30888 raft_consensus.cc:3060] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:02.130476 31270 leader_election.cc:290] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:02.241909 27337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36604 (request call id 690) took 1561 ms. Trace:
I20260807 01:48:02.145874 31117 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:02.316210 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:02.274840 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:02.258339 27337 rpcz_store.cc:276] 0807 01:48:00.680203 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:00.680978 (+ 775us) service_pool.cc:224] Handling call
0807 01:48:00.681332 (+ 354us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:00.681657 (+ 325us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:00.681685 (+ 28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:00.682207 (+ 522us) log.cc:844] Serialized 26 byte log entry
0807 01:48:01.690506 (+1008299us) raft_consensus.cc:1645] Marking committed up to 5
0807 01:48:01.690544 (+ 38us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:01.690610 (+ 66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:02.241662 (+551052us) raft_consensus.cc:1685] finished
0807 01:48:02.241684 (+ 22us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:02.241831 (+ 147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":866414,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":189,"thread_start_us":1007886,"threads_started":1,"wal-append.queue_time_us":1118633}
W20260807 01:48:02.241801 31290 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.018s user 0.013s sys 0.074s
W20260807 01:48:02.268957 28655 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s user 0.008s sys 0.022s
W20260807 01:48:02.322608 28655 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.581s user 0.008s sys 0.025s
I20260807 01:48:02.315493 27333 raft_consensus.cc:1240] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260807 01:48:02.326659 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 585066us
I20260807 01:48:02.242012 31281 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:02.132299 31565 consensus_queue.cc:1048] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.192s
W20260807 01:48:02.340191 28661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:02.322727 27063 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.572s user 0.001s sys 0.021s
I20260807 01:48:02.348527 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:02.345752 27063 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.595s user 0.002s sys 0.021s
W20260807 01:48:02.349648 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 599450us
I20260807 01:48:02.352352 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:02.350433 27063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:02.321053 31589 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:02.133646 31568 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:02.223759 31467 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:02.331564 31587 log.cc:927] Time spent T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.357s user 0.000s sys 0.008s
I20260807 01:48:02.335111 31281 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:02.337352 31203 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:02.450023 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:02.481567 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:02.482795 28660 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.730s user 0.007s sys 0.026s
W20260807 01:48:02.483156 28660 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.730s user 0.007s sys 0.026s
W20260807 01:48:02.483430 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 730869us
I20260807 01:48:02.484287 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
W20260807 01:48:02.485881 27227 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.075s user 0.057s sys 0.157s
W20260807 01:48:02.486317 27227 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.075s user 0.057s sys 0.158s
I20260807 01:48:02.487267 27227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57662 (request call id 2698) took 1295 ms. Trace:
I20260807 01:48:02.487614 27227 rpcz_store.cc:276] 0807 01:48:01.191753 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:01.192024 (+ 271us) service_pool.cc:224] Handling call
0807 01:48:01.192450 (+ 426us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:01.192704 (+ 254us) raft_consensus.cc:1534] Early marking committed up to index 7
0807 01:48:01.410779 (+218075us) log.cc:844] Serialized 14 byte log entry
0807 01:48:02.486865 (+1076086us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:02.486926 (+ 61us) raft_consensus.cc:1645] Marking committed up to 7
0807 01:48:02.486974 (+ 48us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:02.487068 (+ 94us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:02.487202 (+ 134us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1075575,"threads_started":1}
I20260807 01:48:02.491497 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:02.328397 31117 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:02.154856 28707 rpcz_store.cc:276] 0807 01:47:59.996609 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:00.642110 (+645501us) service_pool.cc:224] Handling call
0807 01:48:00.642518 (+ 408us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:00.642748 (+ 230us) raft_consensus.cc:1534] Early marking committed up to index 4
0807 01:48:00.881776 (+239028us) log.cc:844] Serialized 14 byte log entry
0807 01:48:02.042062 (+1160286us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:02.042136 (+ 74us) raft_consensus.cc:1645] Marking committed up to 4
0807 01:48:02.052708 (+ 10572us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:02.072248 (+ 19540us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:02.082336 (+ 10088us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77851,"raft.queue_time_us":76639,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"thread_start_us":934167,"threads_started":1,"wal-append.queue_time_us":1249082}
I20260807 01:48:02.522075 28660 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters: 3478729505b046c2a0051c2d0130873f
W20260807 01:48:02.522406 27063 leader_election.cc:336] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.521s (SENT)
I20260807 01:48:02.527489 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:02.529249 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:02.533891 27063 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:02.466055 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
W20260807 01:48:02.530717 27067 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s user 0.001s sys 0.067s
W20260807 01:48:02.537138 27067 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.664s user 0.001s sys 0.067s
I20260807 01:48:02.539706 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
W20260807 01:48:02.548059 28662 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.769s user 0.003s sys 0.028s
I20260807 01:48:02.485793 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:02.550063 28662 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.771s user 0.003s sys 0.028s
W20260807 01:48:02.542387 31409 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.728s user 0.035s sys 0.033s
I20260807 01:48:02.551409 31484 raft_consensus.cc:2804] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:48:02.563338 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:02.477429 31589 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:02.601886 27337 raft_consensus.cc:3055] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 7 LEADER]: Stepping down as leader of term 7
W20260807 01:48:02.602842 28660 connection.cc:256] RPC call timeout handler was delayed by -1.1145663559436798s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18620000000000003s for a response
I20260807 01:48:02.615131 27337 raft_consensus.cc:740] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:02.571132 27327 raft_consensus.cc:1240] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 2. Current term is 3. Ops: []
W20260807 01:48:02.561542 31570 log.cc:927] Time spent T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.481s user 0.001s sys 0.001s
W20260807 01:48:02.622668 27170 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.553s user 0.005s sys 0.027s
W20260807 01:48:02.624728 27170 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.556s user 0.006s sys 0.028s
W20260807 01:48:02.564028 31409 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.766s user 0.036s sys 0.033s
I20260807 01:48:02.564208 28706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48350 (request call id 835) took 1459 ms. Trace:
W20260807 01:48:02.605652 31581 log.cc:927] Time spent T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.477s user 0.001s sys 0.001s
I20260807 01:48:02.561410 28713 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:48:02.631333 27337 consensus_queue.cc:260] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:02.636320 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 406901us
W20260807 01:48:02.628901 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 560004us
W20260807 01:48:02.653381 27170 leader_election.cc:336] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.885s (SENT)
I20260807 01:48:02.653822 27170 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:02.656047 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:02.651424 26932 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.563s user 0.005s sys 0.018s
W20260807 01:48:02.660550 26932 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.577s user 0.005s sys 0.018s
W20260807 01:48:02.660813 26932 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 577281us
I20260807 01:48:02.667174 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:02.667965 27338 raft_consensus.cc:3060] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:02.621213 31509 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:02.675351 27337 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Advancing to term 8
W20260807 01:48:02.652277 31585 log.cc:927] Time spent T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.412s user 0.000s sys 0.001s
W20260807 01:48:02.650697 31490 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s user 0.012s sys 0.010s
I20260807 01:48:02.629355 31465 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:02.627597 31601 raft_consensus.cc:493] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
W20260807 01:48:02.687152 27170 connection.cc:256] RPC call timeout handler was delayed by -1.2188348099589348s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17390000000000003s for a response
W20260807 01:48:02.744895 27170 leader_election.cc:336] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.561s (SENT)
W20260807 01:48:02.745615 27170 connection.cc:256] RPC call timeout handler was delayed by -1.2098348098807037s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17490000000000003s for a response
W20260807 01:48:02.746037 27170 connection.cc:256] RPC call timeout handler was delayed by -1.1677184940781444s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2046s for a response
W20260807 01:48:02.746431 27170 connection.cc:256] RPC call timeout handler was delayed by -1.0447008078917861s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17700000000000002s for a response
I20260807 01:48:02.687986 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:02.747143 27336 raft_consensus.cc:3060] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:02.688800 27326 raft_consensus.cc:1275] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260807 01:48:02.690472 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:02.691404 27338 raft_consensus.cc:2468] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:48:02.633822 31484 raft_consensus.cc:697] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 6 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:02.679936 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:02.698377 31609 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:48:02.654943 31614 raft_consensus.cc:2749] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:02.764202 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:02.765120 27338 raft_consensus.cc:2368] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 6. Current term is 7.
W20260807 01:48:02.677614 31289 consensus_meta.cc:287] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade: Time spent flushing consensus metadata: real 1.424s user 0.003s sys 0.015s
I20260807 01:48:02.687122 31509 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:02.700244 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:02.701081 31589 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:02.599174 28707 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:52322
W20260807 01:48:02.672427 28713 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48350 (request call id 830) took 1689 ms (client timeout 2243 ms). Trace:
I20260807 01:48:02.702517 31467 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:48:02.703274 31597 log.cc:927] Time spent T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.208s user 0.000s sys 0.000s
W20260807 01:48:02.704092 31599 log.cc:927] Time spent T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.152s user 0.000s sys 0.000s
I20260807 01:48:02.704519 27229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41230 (request call id 479) took 1968 ms. Trace:
W20260807 01:48:02.704885 31594 log.cc:927] Time spent T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.166s user 0.000s sys 0.000s
W20260807 01:48:02.705564 31591 log.cc:927] Time spent T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.294s user 0.000s sys 0.001s
W20260807 01:48:02.705896 31473 consensus_meta.cc:287] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 1.450s user 0.021s sys 0.008s
W20260807 01:48:02.707235 31171 consensus_meta.cc:287] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 1.001s user 0.004s sys 0.006s
W20260807 01:48:02.709512 31503 consensus_meta.cc:287] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.966s user 0.014s sys 0.002s
I20260807 01:48:02.692824 31409 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:02.719483 27110 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 2996) took 2064 ms. Trace:
W20260807 01:48:02.719481 31611 log.cc:927] Time spent T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.099s user 0.000s sys 0.000s
I20260807 01:48:02.860047 27229 rpcz_store.cc:276] 0807 01:48:00.736452 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:00.737762 (+ 1310us) service_pool.cc:224] Handling call
0807 01:48:00.738602 (+ 840us) raft_consensus.cc:1471] Updating replica for 2 ops
0807 01:48:00.743462 (+ 4860us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:00.743490 (+ 28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0807 01:48:00.746620 (+ 3130us) log.cc:844] Serialized 52 byte log entry
0807 01:48:01.720135 (+973515us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:48:02.199246 (+479111us) log.cc:844] Serialized 14 byte log entry
0807 01:48:02.199519 (+ 273us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:02.199615 (+ 96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:02.704230 (+504615us) raft_consensus.cc:1685] finished
0807 01:48:02.704253 (+ 23us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:02.704445 (+ 192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89931,"prepare.queue_time_us":798750,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":9637,"raft.queue_time_us":5214,"raft.run_cpu_time_us":431,"raft.run_wall_time_us":20060,"thread_start_us":1421426,"threads_started":2,"wal-append.queue_time_us":1102247}
W20260807 01:48:02.693869 31490 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.612s user 0.013s sys 0.011s
I20260807 01:48:02.726902 31568 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:02.728302 31281 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:02.729259 31117 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:48:02.733275 31566 log.cc:927] Time spent T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.947s user 0.004s sys 0.000s
W20260807 01:48:02.734421 31584 log.cc:927] Time spent T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.402s user 0.001s sys 0.000s
I20260807 01:48:02.734947 27325 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
W20260807 01:48:02.737879 31190 consensus_meta.cc:287] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 1.047s user 0.007s sys 0.003s
I20260807 01:48:02.633744 28706 rpcz_store.cc:276] 0807 01:48:01.102425 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:01.105305 (+ 2880us) service_pool.cc:224] Handling call
0807 01:48:01.105910 (+ 605us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:01.110790 (+ 4880us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:01.110839 (+ 49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:01.111443 (+ 604us) log.cc:844] Serialized 26 byte log entry
0807 01:48:01.821733 (+710290us) raft_consensus.cc:1645] Marking committed up to 5
0807 01:48:01.890137 (+ 68404us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:01.911551 (+ 21414us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:02.546602 (+635051us) raft_consensus.cc:1685] finished
0807 01:48:02.546635 (+ 33us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:02.561255 (+ 14620us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2542,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":135,"thread_start_us":690655,"threads_started":1,"wal-append.queue_time_us":688998}
I20260807 01:48:02.740039 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:48:02.691581 31598 log.cc:927] Time spent T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.411s user 0.000s sys 0.001s
I20260807 01:48:02.687253 31420 raft_consensus.cc:2749] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260807 01:48:02.742549 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:02.743292 27230 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s user 0.026s sys 0.047s
W20260807 01:48:02.754491 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:02.961999 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:02.962708 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:02.760531 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
W20260807 01:48:02.961539 27230 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.737s user 0.026s sys 0.047s
W20260807 01:48:02.774199 27063 leader_election.cc:336] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.853s (SENT)
I20260807 01:48:02.964154 27230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57662 (request call id 2710) took 1619 ms. Trace:
I20260807 01:48:02.837390 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:02.838980 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:02.841825 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:02.750349 31511 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:02.752770 31203 raft_consensus.cc:493] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:02.868852 27110 rpcz_store.cc:276] 0807 01:48:00.655140 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:00.655285 (+ 145us) service_pool.cc:224] Handling call
0807 01:48:00.655636 (+ 351us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:00.662401 (+ 6765us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:00.662431 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:00.903631 (+241200us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b2c000948d8. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d53d19c71 00007f7d53d295d2 00007f7d55459de1 00007f7d5541e81a 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22
0807 01:48:00.904309 (+ 678us) log.cc:844] Serialized 26 byte log entry
0807 01:48:01.789427 (+885118us) raft_consensus.cc:1645] Marking committed up to 5
0807 01:48:01.789469 (+ 42us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:01.789555 (+ 86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:02.719129 (+929574us) raft_consensus.cc:1685] finished
0807 01:48:02.719150 (+ 21us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:02.719402 (+ 252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6368,"raft.queue_time_us":6762,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":6011,"spinlock_wait_cycles":5588864,"thread_start_us":884598,"threads_started":1,"wal-append.queue_time_us":1069959}
I20260807 01:48:03.190704 27327 raft_consensus.cc:2393] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 5: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
W20260807 01:48:02.721762 31003 consensus_meta.cc:287] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade: Time spent flushing consensus metadata: real 0.921s user 0.006s sys 0.006s
I20260807 01:48:03.194334 31511 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:02.915756 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 259651us
I20260807 01:48:02.917121 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:02.865049 28707 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:48:02.753224 31465 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:02.927829 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:02.941126 28709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48350 (request call id 832) took 1929 ms. Trace:
W20260807 01:48:02.949647 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:02.956470 31484 consensus_queue.cc:237] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:03.168912 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 115679us
I20260807 01:48:03.020695 27063 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:02.914319 31409 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:03.096282 27212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41230 (request call id 478) took 2367 ms. Trace:
I20260807 01:48:03.168358 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:48:03.204764 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:03.205583 27227 raft_consensus.cc:3060] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:48:03.207526 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:03.208274 27214 raft_consensus.cc:3060] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:03.209199 27218 raft_consensus.cc:1240] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5. Ops: []
W20260807 01:48:02.838973 28713 rpcz_store.cc:269] 0807 01:48:00.980195 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:00.984390 (+ 4195us) service_pool.cc:224] Handling call
0807 01:48:02.479656 (+1495266us) spinlock_profiling.cc:243] Waited 9.09 ms on lock 0x7b5800055598. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e0521 00007f7d522e089e 000000000038d209 00007f7d5a39f940 00007f7d5a467ed4 00007f7d5a467f41 00007f7d5a415eb0 00007f7d5a415794 00007f7d5395e834 00007f7d5395e7c2
0807 01:48:02.669493 (+189837us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20899200}
W20260807 01:48:03.091537 31619 log.cc:927] Time spent T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.083s user 0.000s sys 0.000s
I20260807 01:48:03.044399 28710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48350 (request call id 836) took 1911 ms. Trace:
I20260807 01:48:03.224473 28710 rpcz_store.cc:276] 0807 01:48:01.133422 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:01.184981 (+ 51559us) service_pool.cc:224] Handling call
0807 01:48:01.185401 (+ 420us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:01.185850 (+ 449us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:01.185878 (+ 28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:01.186607 (+ 729us) log.cc:844] Serialized 26 byte log entry
0807 01:48:02.086329 (+899722us) raft_consensus.cc:1645] Marking committed up to 4
0807 01:48:02.086368 (+ 39us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:02.086437 (+ 69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:03.044131 (+957694us) raft_consensus.cc:1685] finished
0807 01:48:03.044154 (+ 23us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:03.044324 (+ 170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":160,"thread_start_us":899145,"threads_started":1,"wal-append.queue_time_us":1181804}
I20260807 01:48:03.042994 27326 raft_consensus.cc:2368] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 3. Current term is 4.
I20260807 01:48:03.078612 31281 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:03.041399 31589 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:03.233373 31589 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:02.829156 31609 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:03.234369 31289 raft_consensus.cc:493] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:03.234956 31289 raft_consensus.cc:515] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:03.236469 31609 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:03.055764 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.210525 31203 raft_consensus.cc:515] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:03.238286 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:03.239442 31203 leader_election.cc:290] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:02.770241 31601 raft_consensus.cc:515] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:02.908335 31503 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:03.180536 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:03.097738 31509 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:03.021324 31622 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:03.071108 31620 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:48:03.071108 31563 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:03.091022 31614 raft_consensus.cc:493] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:03.022302 31627 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:03.083818 30888 consensus_queue.cc:1059] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260807 01:48:03.078344 28714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48350 (request call id 833) took 2063 ms. Trace:
I20260807 01:48:02.964707 27230 rpcz_store.cc:276] 0807 01:48:01.344874 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:01.347159 (+ 2285us) service_pool.cc:224] Handling call
0807 01:48:01.350266 (+ 3107us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:01.350505 (+ 239us) raft_consensus.cc:1534] Early marking committed up to index 4
0807 01:48:02.224086 (+873581us) log.cc:844] Serialized 14 byte log entry
0807 01:48:02.963757 (+739671us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:02.963818 (+ 61us) raft_consensus.cc:1645] Marking committed up to 4
0807 01:48:02.963872 (+ 54us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:02.963950 (+ 78us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:02.964086 (+ 136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1015518,"prepare.run_cpu_time_us":18751,"prepare.run_wall_time_us":101138,"thread_start_us":1559760,"threads_started":2}
I20260807 01:48:03.202916 31269 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:03.183064 27336 raft_consensus.cc:2468] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:48:03.052690 31565 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:03.168358 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.251793 28714 rpcz_store.cc:276] 0807 01:48:01.015389 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:01.015539 (+ 150us) service_pool.cc:224] Handling call
0807 01:48:01.040149 (+ 24610us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:01.076405 (+ 36256us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:01.076443 (+ 38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:01.080891 (+ 4448us) log.cc:844] Serialized 26 byte log entry
0807 01:48:01.807658 (+726767us) raft_consensus.cc:1645] Marking committed up to 5
0807 01:48:01.807747 (+ 89us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:02.065534 (+257787us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b54004f3654. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:02.065600 (+ 66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:03.068427 (+1002827us) raft_consensus.cc:1685] finished
0807 01:48:03.068454 (+ 27us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:03.078271 (+ 9817us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6895,"raft.queue_time_us":12491,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":11015,"spinlock_wait_cycles":41243520,"thread_start_us":726035,"threads_started":1,"wal-append.queue_time_us":1168502}
I20260807 01:48:03.259032 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:03.259783 27337 raft_consensus.cc:3060] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
W20260807 01:48:03.186687 27170 connection.cc:256] RPC call timeout handler was delayed by -1.1152687191497535s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1593s for a response
W20260807 01:48:03.260679 27170 connection.cc:256] RPC call timeout handler was delayed by -1.1125603220425546s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2036s for a response
W20260807 01:48:03.261034 27170 connection.cc:256] RPC call timeout handler was delayed by -1.0513603219296783s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21040000000000003s for a response
W20260807 01:48:03.261518 27170 leader_election.cc:336] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.692s (SENT)
W20260807 01:48:03.262444 27170 leader_election.cc:336] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.739s (SENT)
W20260807 01:48:03.263278 27170 leader_election.cc:336] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.749s (SENT)
I20260807 01:48:03.263666 27170 leader_election.cc:304] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
W20260807 01:48:03.264879 27170 leader_election.cc:336] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.770s (SENT)
I20260807 01:48:03.265306 27170 leader_election.cc:304] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:03.189507 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:03.209084 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:03.203717 27212 rpcz_store.cc:276] 0807 01:48:00.729105 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:00.729207 (+ 102us) service_pool.cc:224] Handling call
0807 01:48:00.884126 (+154919us) raft_consensus.cc:1471] Updating replica for 2 ops
0807 01:48:00.973779 (+ 89653us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:00.973807 (+ 28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0807 01:48:01.000867 (+ 27060us) log.cc:844] Serialized 52 byte log entry
0807 01:48:01.749111 (+748244us) raft_consensus.cc:1645] Marking committed up to 5
0807 01:48:02.282023 (+532912us) log.cc:844] Serialized 14 byte log entry
0807 01:48:02.285354 (+ 3331us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:02.285442 (+ 88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:03.095889 (+810447us) raft_consensus.cc:1685] finished
0807 01:48:03.095915 (+ 26us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:03.096196 (+ 281us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100974,"prepare.queue_time_us":809673,"prepare.run_cpu_time_us":239,"prepare.run_wall_time_us":29764,"raft.queue_time_us":110991,"raft.run_cpu_time_us":420,"raft.run_wall_time_us":27040,"thread_start_us":1185050,"threads_started":2,"wal-append.queue_time_us":1137194}
I20260807 01:48:03.200592 28709 rpcz_store.cc:276] 0807 01:48:01.011805 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:01.011974 (+ 169us) service_pool.cc:224] Handling call
0807 01:48:01.012331 (+ 357us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:01.012564 (+ 233us) raft_consensus.cc:1534] Early marking committed up to index 6
0807 01:48:01.759706 (+747142us) log.cc:844] Serialized 14 byte log entry
0807 01:48:02.940368 (+1180662us) spinlock_profiling.cc:243] Waited 4.7 ms on lock 0x7b180033d240. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d5b233f31 00007f7d5b2406d1 00007f7d54fc56ac 00007f7d522fae3b 00007f7d5b1a2fc9 00007f7d52316fdb 00007f7d523152db 00007f7d52317542 00007f7d54fb73fb 00007f7d54fbcf13 00007f7d54fbd42a
0807 01:48:02.940661 (+ 293us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:02.940733 (+ 72us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:48:02.940798 (+ 65us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:02.940880 (+ 82us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:02.941042 (+ 162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":98715,"prepare.queue_time_us":1120622,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":53,"raft.queue_time_us":1239,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":118,"spinlock_wait_cycles":10818944,"thread_start_us":1123998,"threads_started":2,"wal-append.queue_time_us":791921}
I20260807 01:48:03.220830 27227 raft_consensus.cc:2468] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:48:03.222849 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:03.223045 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:03.223366 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.267557 27327 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:03.228334 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.231854 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:03.232589 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:03.243584 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:03.268791 27213 raft_consensus.cc:2468] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:48:03.043920 31568 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:03.267666 27226 raft_consensus.cc:2468] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:48:03.197767 31420 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:03.210740 28707 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:03.256528 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.266356 27338 raft_consensus.cc:2368] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6.
I20260807 01:48:03.273999 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.274907 27338 raft_consensus.cc:2410] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260807 01:48:03.275144 27170 leader_election.cc:336] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.813s (SENT)
I20260807 01:48:03.275743 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:03.268343 27333 raft_consensus.cc:2410] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260807 01:48:03.276299 27170 leader_election.cc:336] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.817s (SENT)
I20260807 01:48:03.276795 27170 leader_election.cc:304] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:03.277209 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:03.277971 27333 raft_consensus.cc:2393] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 3: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:03.279417 27331 raft_consensus.cc:2393] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 5: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:03.279194 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
W20260807 01:48:03.288892 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 92092.5us
I20260807 01:48:03.251463 28702 raft_consensus.cc:2410] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260807 01:48:03.277935 27170 leader_election.cc:336] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.830s (SENT)
I20260807 01:48:03.309074 31416 raft_consensus.cc:2749] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:48:03.309585 27222 raft_consensus.cc:2393] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 5: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:03.311810 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.315824 27170 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:03.309649 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:03.348901 27326 raft_consensus.cc:3060] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
W20260807 01:48:03.324095 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 46193.2us
W20260807 01:48:03.350091 27170 leader_election.cc:336] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.835s (SENT)
I20260807 01:48:03.350566 27170 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
W20260807 01:48:03.351707 27170 leader_election.cc:336] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.844s (SENT)
I20260807 01:48:03.352095 27170 leader_election.cc:304] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:03.311416 31568 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:03.312860 27337 raft_consensus.cc:2468] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:48:03.356316 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.357197 27337 raft_consensus.cc:2393] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 8: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:03.358050 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.359447 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:03.360790 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.322455 27333 raft_consensus.cc:2368] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 6. Current term is 7.
I20260807 01:48:03.279965 31458 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:03.327018 31473 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:48:03.254909 31627 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:03.330855 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:03.313634 31420 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:03.332394 27214 raft_consensus.cc:2410] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260807 01:48:03.332466 31409 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:03.334481 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.334141 27327 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:48:03.380036 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.342077 31637 raft_consensus.cc:2749] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:03.380856 27327 raft_consensus.cc:2393] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 8: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:03.335186 31465 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:03.258533 31622 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:03.319444 31511 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:03.281368 31203 raft_consensus.cc:2749] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:48:03.281944 31565 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:03.268010 31269 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:03.343406 31632 raft_consensus.cc:493] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:03.344154 31628 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:03.345005 31281 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:03.252311 31614 raft_consensus.cc:515] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:03.310216 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:03.362617 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.279858 31500 raft_consensus.cc:2749] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:48:03.320425 31650 consensus_queue.cc:1059] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260807 01:48:03.378590 31473 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:03.364394 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 590236us
I20260807 01:48:03.366628 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.381361 27331 raft_consensus.cc:2368] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 6. Current term is 7.
I20260807 01:48:03.249219 31620 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:03.386077 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.248647 31563 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:03.372435 31458 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:03.447121 27110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:41078
W20260807 01:48:03.440555 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 126631us
I20260807 01:48:03.412472 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.468565 27327 raft_consensus.cc:2376] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5. Re-sending same reply.
I20260807 01:48:03.422676 27336 raft_consensus.cc:2368] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 10. Current term is 11.
I20260807 01:48:03.471931 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:48:03.460676 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 108969us
I20260807 01:48:03.469951 27110 raft_consensus.cc:606] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Received request to transfer leadership
W20260807 01:48:03.483246 27170 leader_election.cc:336] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.845s (SENT)
I20260807 01:48:03.484064 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.487234 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.485162 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:03.485838 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:03.488422 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:48:03.469175 27063 leader_election.cc:336] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.912s (SENT)
I20260807 01:48:03.489087 31454 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:03.489550 27063 leader_election.cc:336] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.995s (SENT)
I20260807 01:48:03.489645 31454 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:03.490036 27063 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:03.490947 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.491545 27327 raft_consensus.cc:2376] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5. Re-sending same reply.
I20260807 01:48:03.491950 31454 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:03.492473 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.493297 27327 raft_consensus.cc:2410] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260807 01:48:03.494288 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.494804 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:48:03.396307 31492 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260807 01:48:03.495393 27325 raft_consensus.cc:2468] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:48:03.496628 31306 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:03.497711 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:03.498303 27331 raft_consensus.cc:1240] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 3. Current term is 4. Ops: []
I20260807 01:48:03.497198 31306 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:03.499738 28708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52322 (request call id 680) took 1063 ms. Trace:
W20260807 01:48:03.500466 27063 leader_election.cc:336] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.620s (SENT)
I20260807 01:48:03.500898 27063 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:03.499684 27331 raft_consensus.cc:1275] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 01:48:03.501947 31306 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:03.500272 28709 raft_consensus.cc:1240] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 3. Current term is 4. Ops: []
I20260807 01:48:03.503275 31660 raft_consensus.cc:2749] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:03.504129 31660 raft_consensus.cc:2749] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:03.504482 28702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52322 (request call id 676) took 1173 ms. Trace:
I20260807 01:48:03.441056 31649 raft_consensus.cc:1064] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
W20260807 01:48:03.505234 31649 raft_consensus.cc:1068] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade: Unable to promote non-voter 57bb4985157447f0a3ab680fa0026f12: Illegal state: Leader has not yet committed an operation in its own term
I20260807 01:48:03.505699 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:03.506240 27063 leader_election.cc:336] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.649s (SENT)
I20260807 01:48:03.506693 27063 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
W20260807 01:48:03.409384 31596 log.cc:927] Time spent T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 1.065s user 0.003s sys 0.007s
I20260807 01:48:03.507558 31657 raft_consensus.cc:2749] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 01:48:03.483680 31281 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:03.506644 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:03.510890 27063 leader_election.cc:336] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.682s (SENT)
W20260807 01:48:03.512356 27063 leader_election.cc:336] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 2.225s (SENT)
I20260807 01:48:03.513293 27063 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:03.516450 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:03.502789 27331 raft_consensus.cc:1240] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 6. Current term is 7. Ops: []
I20260807 01:48:03.500139 28708 rpcz_store.cc:276] 0807 01:48:02.436660 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:02.558798 (+122138us) service_pool.cc:224] Handling call
0807 01:48:02.653164 (+ 94366us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:02.653396 (+ 232us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:02.653424 (+ 28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:02.653475 (+ 51us) raft_consensus.cc:1645] Marking committed up to 9
0807 01:48:02.653510 (+ 35us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:02.653572 (+ 62us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:03.499556 (+845984us) spinlock_profiling.cc:243] Waited 94 ms on lock 0x7b54008516d0. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541dc8d 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:03.499673 (+ 117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":216112512}
I20260807 01:48:03.521665 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:03.520809 27101 raft_consensus.cc:1240] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 3. Current term is 4. Ops: []
I20260807 01:48:03.524881 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:03.424034 31652 raft_consensus.cc:993] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:48:03.526351 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:03.526978 27227 raft_consensus.cc:3060] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:03.530454 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:48:03.484401 27170 leader_election.cc:336] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.941s (SENT)
I20260807 01:48:03.533270 27227 raft_consensus.cc:2468] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:48:03.535434 27101 raft_consensus.cc:1240] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 2. Current term is 3. Ops: []
W20260807 01:48:03.536233 27063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:03.401665 31617 log.cc:927] Time spent T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.440s user 0.001s sys 0.001s
I20260807 01:48:03.544718 28714 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEARNER]: Advancing to term 5
I20260807 01:48:03.533731 27170 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:03.538406 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:03.547765 27230 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:48:03.541803 27331 raft_consensus.cc:1275] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 01:48:03.494035 31289 leader_election.cc:290] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:03.397202 31562 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:03.424726 31568 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:03.391441 31656 raft_consensus.cc:2749] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 01:48:03.538996 27101 raft_consensus.cc:1240] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5. Ops: []
I20260807 01:48:03.432598 31622 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:03.393432 31416 raft_consensus.cc:2749] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:03.406868 31187 raft_consensus.cc:493] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:03.474484 31503 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:03.549962 27108 raft_consensus.cc:1240] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5. Ops: []
I20260807 01:48:03.550005 27216 raft_consensus.cc:1240] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 2. Current term is 3. Ops: []
W20260807 01:48:03.550947 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:03.551565 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:03.557696 27330 raft_consensus.cc:2468] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:48:03.559633 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:03.572290 27110 raft_consensus.cc:2368] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 5. Current term is 6.
I20260807 01:48:03.543969 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:03.474799 31589 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:03.560546 27333 raft_consensus.cc:1240] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6. Ops: []
I20260807 01:48:03.573702 27338 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 8.
I20260807 01:48:03.509775 31281 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:03.504901 28702 rpcz_store.cc:276] 0807 01:48:02.331426 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:02.334279 (+ 2853us) service_pool.cc:224] Handling call
0807 01:48:03.504276 (+1169997us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b54003e9ed4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:03.504416 (+ 140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":978365824}
W20260807 01:48:03.563665 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 79285us
I20260807 01:48:03.431561 31642 raft_consensus.cc:2749] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:48:03.485965 31628 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:03.431823 31654 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:03.574231 27327 raft_consensus.cc:1275] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 01:48:03.571820 27230 raft_consensus.cc:3060] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:03.575780 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:03.576442 27170 leader_election.cc:336] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.944s (SENT)
I20260807 01:48:03.518082 31632 raft_consensus.cc:515] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:03.578724 31187 raft_consensus.cc:515] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:03.580057 31614 leader_election.cc:290] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:03.580703 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:03.564505 31650 consensus_queue.cc:1243] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer 57bb4985157447f0a3ab680fa0026f12 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260807 01:48:03.559125 31474 raft_consensus.cc:1081] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Signalling peer 57bb4985157447f0a3ab680fa0026f12 to start an election
I20260807 01:48:03.544171 31601 leader_election.cc:290] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:03.581676 31620 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:03.582211 30888 raft_consensus.cc:493] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:03.582530 31458 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:03.582938 31562 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:03.583703 31479 consensus_queue.cc:1059] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 01:48:03.584649 31473 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:03.538254 28707 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:03.571770 31477 raft_consensus.cc:2749] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260807 01:48:03.587998 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:03.590615 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:03.617937 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:03.597191 31281 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:03.565013 31652 raft_consensus.cc:2749] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:03.593422 27101 raft_consensus.cc:2368] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 6. Current term is 7.
I20260807 01:48:03.595899 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:03.597550 27287 connection.cc:256] RPC call timeout handler was delayed by -3.4124220551457256s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1501s for a response
W20260807 01:48:03.623245 27287 connection.cc:256] RPC call timeout handler was delayed by -3.2162220550235361s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17190000000000003s for a response
I20260807 01:48:03.623308 27105 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 7.
W20260807 01:48:03.623680 27287 connection.cc:256] RPC call timeout handler was delayed by -3.1271220550406724s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18180000000000002s for a response
W20260807 01:48:03.624037 27287 connection.cc:256] RPC call timeout handler was delayed by -2.7689220551401377s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22160000000000002s for a response
I20260807 01:48:03.601907 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:03.598750 31589 raft_consensus.cc:493] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:03.613961 27326 raft_consensus.cc:2468] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:48:03.600194 31269 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:03.615265 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:03.588245 31409 consensus_queue.cc:1059] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260807 01:48:03.616405 31650 raft_consensus.cc:3055] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:48:03.616012 30888 raft_consensus.cc:515] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:03.629971 31650 raft_consensus.cc:740] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:03.631534 30888 leader_election.cc:290] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:03.631129 31650 consensus_queue.cc:260] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:03.621569 31635 log.cc:927] Time spent T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.116s user 0.000s sys 0.000s
I20260807 01:48:03.599332 28714 raft_consensus.cc:1275] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEARNER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260807 01:48:03.624330 27287 connection.cc:256] RPC call timeout handler was delayed by -2.7518220550846308s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2235s for a response
I20260807 01:48:03.627103 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.605432 31420 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:03.591133 27170 leader_election.cc:304] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:03.616894 31565 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:03.597689 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:03.617725 31627 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:03.647732 27333 raft_consensus.cc:2393] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 4: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:03.648737 27110 raft_consensus.cc:2468] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
W20260807 01:48:03.647567 27287 connection.cc:256] RPC call timeout handler was delayed by -2.3999220551922917s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2626s for a response
I20260807 01:48:03.651016 27110 raft_consensus.cc:1240] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260807 01:48:03.648222 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 71818.6us
I20260807 01:48:03.651690 27287 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
W20260807 01:48:03.652356 27170 leader_election.cc:336] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 2.046s (SENT)
I20260807 01:48:03.652798 27170 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:03.653833 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.657150 27287 leader_election.cc:304] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:03.633447 31396 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:03.654579 27230 raft_consensus.cc:2468] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:48:03.658654 27333 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
I20260807 01:48:03.639009 28702 raft_consensus.cc:1240] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEARNER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5. Ops: []
I20260807 01:48:03.641886 31651 raft_consensus.cc:3055] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:48:03.635735 31576 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:03.667972 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.643649 31187 leader_election.cc:290] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:03.657330 31565 consensus_queue.cc:1059] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260807 01:48:03.673221 27287 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:03.652724 28706 raft_consensus.cc:1240] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 6. Current term is 7. Ops: []
I20260807 01:48:03.638266 31563 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:03.653842 31477 raft_consensus.cc:2749] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:03.599603 31654 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:03.682433 31161 consensus_queue.cc:1048] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260807 01:48:03.685586 30888 raft_consensus.cc:3055] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Stepping down as leader of term 6
I20260807 01:48:03.668844 31420 raft_consensus.cc:2749] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:48:03.687124 27287 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:03.687212 31563 raft_consensus.cc:2749] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:03.687556 31654 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:03.688650 31563 raft_consensus.cc:2749] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260807 01:48:03.662654 31650 raft_consensus.cc:3060] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:03.695309 27328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36604 (request call id 707) took 1029 ms. Trace:
I20260807 01:48:03.696722 27287 leader_election.cc:304] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:03.676199 28655 leader_election.cc:304] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:03.698815 28655 leader_election.cc:304] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:03.653443 28713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:34614
I20260807 01:48:03.701040 28709 raft_consensus.cc:1240] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5. Ops: []
I20260807 01:48:03.629460 31281 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:03.695796 27328 rpcz_store.cc:276] 0807 01:48:02.666319 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:02.666430 (+ 111us) service_pool.cc:224] Handling call
0807 01:48:02.689490 (+ 23060us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:02.689948 (+ 458us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:02.689978 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:02.718537 (+ 28559us) log.cc:844] Serialized 26 byte log entry
0807 01:48:03.041983 (+323446us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:48:03.042032 (+ 49us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:03.042110 (+ 78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:03.695065 (+652955us) raft_consensus.cc:1685] finished
0807 01:48:03.695090 (+ 25us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:03.695240 (+ 150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4175,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"thread_start_us":322963,"threads_started":1,"wal-append.queue_time_us":509386}
I20260807 01:48:03.657218 31658 raft_consensus.cc:2749] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 01:48:03.714632 31161 raft_consensus.cc:2804] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:48:03.715378 31161 raft_consensus.cc:697] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:03.716351 31161 consensus_queue.cc:237] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:03.716900 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 288495us
I20260807 01:48:03.639202 31589 raft_consensus.cc:515] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:03.711719 31590 raft_consensus.cc:2764] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260807 01:48:03.719990 31589 leader_election.cc:290] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:03.676779 31651 raft_consensus.cc:740] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:03.721400 31651 consensus_queue.cc:260] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:03.702277 31576 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:03.722739 31651 raft_consensus.cc:3060] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:48:03.724009 31576 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:03.724301 31562 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:03.714597 31306 raft_consensus.cc:2804] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:48:03.725066 31306 raft_consensus.cc:697] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 6 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:03.727684 31281 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:03.728482 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:03.728674 31628 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:03.728968 31632 leader_election.cc:290] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:03.676403 31396 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:03.730245 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:03.730500 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:03.656726 31568 consensus_queue.cc:1048] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.075s
I20260807 01:48:03.731665 31396 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:03.732452 31509 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:03.726011 31306 consensus_queue.cc:237] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:03.733767 31187 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:03.731560 28713 raft_consensus.cc:606] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 7 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:48:03.734314 31187 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:03.735188 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 82854.2us
I20260807 01:48:03.686020 30888 raft_consensus.cc:740] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:03.675595 31492 raft_consensus.cc:2749] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 01:48:03.727746 28707 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:03.728021 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:03.735647 31658 consensus_queue.cc:1059] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260807 01:48:03.739075 27336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33976 (request call id 2539) took 1029 ms. Trace:
I20260807 01:48:03.739480 27336 rpcz_store.cc:276] 0807 01:48:02.709354 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:03.312161 (+602807us) service_pool.cc:224] Handling call
0807 01:48:03.738876 (+426715us) spinlock_profiling.cc:243] Waited 52.8 ms on lock 0x7b540027bfd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:03.739013 (+ 137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":121448704}
I20260807 01:48:03.740062 28711 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:34614
I20260807 01:48:03.730257 31632 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:03.740687 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:03.736565 28707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:52322 (request call id 685) took 1175 ms. Trace:
I20260807 01:48:03.740711 28711 raft_consensus.cc:606] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:48:03.743034 28707 rpcz_store.cc:276] 0807 01:48:02.561706 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:02.563178 (+ 1472us) service_pool.cc:224] Handling call
0807 01:48:03.736471 (+1173293us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 01:48:03.735872 27170 leader_election.cc:336] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 2.088s (SENT)
I20260807 01:48:03.745280 27170 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:03.746243 31477 raft_consensus.cc:2749] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260807 01:48:03.747340 27170 leader_election.cc:336] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 2.121s (SENT)
I20260807 01:48:03.747768 27170 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:03.750434 31477 raft_consensus.cc:2749] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:48:03.740911 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:03.752727 27170 leader_election.cc:336] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 2.167s (SENT)
I20260807 01:48:03.753669 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:03.754686 27101 raft_consensus.cc:2410] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260807 01:48:03.733052 31509 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:03.733357 27327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33976 (request call id 2529) took 1026 ms. Trace:
I20260807 01:48:03.764669 27327 rpcz_store.cc:276] 0807 01:48:02.706508 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:03.495182 (+788674us) service_pool.cc:224] Handling call
0807 01:48:03.573864 (+ 78682us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:03.589383 (+ 15519us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:03.733148 (+143765us) spinlock_profiling.cc:243] Waited 78.3 ms on lock 0x7b540007eed0. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541dc8d 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:03.733297 (+ 149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":180188928}
I20260807 01:48:03.741017 31632 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:03.721735 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:03.757535 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:03.758354 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:03.766969 27230 raft_consensus.cc:2468] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 7.
I20260807 01:48:03.758764 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:03.768905 27229 raft_consensus.cc:2393] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 5: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:03.697180 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:48:03.760811 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 55383.4us
I20260807 01:48:03.703091 28655 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:03.765443 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:03.773484 27105 raft_consensus.cc:2468] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:48:03.774914 31576 raft_consensus.cc:2804] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:03.776000 31576 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:03.776401 31576 raft_consensus.cc:3060] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
W20260807 01:48:03.781055 27063 leader_election.cc:336] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 2.175s (SENT)
I20260807 01:48:03.781574 27063 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
W20260807 01:48:03.784489 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:03.786396 31627 raft_consensus.cc:2749] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260807 01:48:03.780582 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 77552.2us
I20260807 01:48:03.795590 31589 consensus_queue.cc:1059] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:48:03.806222 31290 consensus_queue.cc:1059] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:48:03.807376 31290 consensus_queue.cc:1243] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer 3478729505b046c2a0051c2d0130873f log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260807 01:48:03.807866 31563 raft_consensus.cc:3055] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:48:03.808411 31563 raft_consensus.cc:740] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
W20260807 01:48:03.808681 27063 leader_election.cc:336] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 2.443s (SENT)
I20260807 01:48:03.808054 31589 consensus_queue.cc:1243] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer 3478729505b046c2a0051c2d0130873f log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260807 01:48:03.810269 31658 raft_consensus.cc:3055] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:48:03.809471 31563 consensus_queue.cc:260] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:03.810704 31658 raft_consensus.cc:740] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:03.811836 31658 consensus_queue.cc:260] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:48:03.813460 31658 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:03.822328 27287 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
W20260807 01:48:03.830128 27063 leader_election.cc:336] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 2.622s (SENT)
I20260807 01:48:03.831717 27108 raft_consensus.cc:1275] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260807 01:48:03.832356 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:03.835696 27222 raft_consensus.cc:2468] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:48:03.836018 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:03.833037 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:03.833992 28711 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b510849c3131475d8c1e032378dc893b"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:34614
I20260807 01:48:03.836609 27213 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
W20260807 01:48:03.837023 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 68553.4us
I20260807 01:48:03.736601 31187 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:03.840368 27287 leader_election.cc:304] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:03.841719 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:03.845084 27109 raft_consensus.cc:1240] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 3. Current term is 4. Ops: []
I20260807 01:48:03.845225 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:03.846382 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 93685us
I20260807 01:48:03.847127 27110 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:03.848556 27108 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:52556
I20260807 01:48:03.849126 27108 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:48:03.849466 27108 raft_consensus.cc:3060] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:03.850435 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:03.850998 27102 raft_consensus.cc:3060] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 2 FOLLOWER]: Advancing to term 4
W20260807 01:48:03.890955 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:03.903234 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:03.891917 27100 consensus_queue.cc:237] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: false } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:03.903967 31582 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:03.911408 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:03.912197 27212 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
I20260807 01:48:03.914121 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:03.839046 31642 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:03.908169 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:03.838784 31589 raft_consensus.cc:2749] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:03.926184 27227 raft_consensus.cc:1240] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 4. Current term is 5. Ops: []
I20260807 01:48:03.840950 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:03.958739 27213 raft_consensus.cc:2468] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:48:03.962165 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
W20260807 01:48:04.004635 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:04.018378 27227 raft_consensus.cc:3060] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
W20260807 01:48:04.026454 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:04.010542 27102 raft_consensus.cc:2468] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:48:04.031178 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:03.831357 31458 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:04.037676 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:48:04.036242 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 196271us
I20260807 01:48:03.904675 31582 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:03.949954 31642 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:03.924254 31576 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:04.047160 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:04.580672 27214 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
W20260807 01:48:04.527328 27063 leader_election.cc:336] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.753s (SENT)
W20260807 01:48:04.127858 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:04.582145 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 54886.5us
W20260807 01:48:04.539024 27067 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.918s user 0.011s sys 0.034s
W20260807 01:48:04.590265 27067 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.969s user 0.013s sys 0.034s
W20260807 01:48:04.590698 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 970069us
I20260807 01:48:04.091884 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:04.074553 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:04.539695 27330 raft_consensus.cc:3055] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:48:04.593739 27330 raft_consensus.cc:740] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
W20260807 01:48:04.440086 27110 consensus_meta.cc:287] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.592s user 0.007s sys 0.000s
I20260807 01:48:04.588335 27287 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:04.193288 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
W20260807 01:48:04.530544 27105 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s user 0.012s sys 0.033s
W20260807 01:48:04.596586 27105 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s user 0.013s sys 0.033s
I20260807 01:48:04.596746 27212 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:04.457571 27227 raft_consensus.cc:2468] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
W20260807 01:48:04.583412 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:04.597106 27287 leader_election.cc:304] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:04.597781 27110 raft_consensus.cc:2468] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
W20260807 01:48:04.606375 27175 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s user 0.013s sys 0.037s
W20260807 01:48:04.606987 27175 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s user 0.013s sys 0.037s
W20260807 01:48:04.607335 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 690207us
I20260807 01:48:04.609453 27175 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
W20260807 01:48:04.609908 27170 leader_election.cc:336] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 2.036s (SENT)
W20260807 01:48:04.611114 27170 leader_election.cc:336] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 2.036s (SENT)
I20260807 01:48:04.090356 27108 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:04.343040 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:04.627665 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:04.631212 27170 leader_election.cc:336] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 2.041s (SENT)
W20260807 01:48:04.487036 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:04.324783 27230 raft_consensus.cc:1275] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260807 01:48:04.628880 28660 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.541s user 0.008s sys 0.001s
W20260807 01:48:04.635246 28660 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.547s user 0.008s sys 0.001s
W20260807 01:48:04.635644 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 547826us
W20260807 01:48:04.633275 27170 leader_election.cc:336] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 2.846s (SENT)
I20260807 01:48:04.638581 27170 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:04.594936 27330 consensus_queue.cc:260] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:04.028786 31484 raft_consensus.cc:493] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
W20260807 01:48:04.533558 31691 log.cc:927] Time spent T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.286s user 0.001s sys 0.000s
I20260807 01:48:04.656893 27108 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:04.654955 27212 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
I20260807 01:48:04.659680 27330 raft_consensus.cc:3060] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:04.044574 31492 raft_consensus.cc:2804] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:48:04.661036 27213 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:48:04.409971 28706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52322 (request call id 688) took 1779 ms. Trace:
W20260807 01:48:04.629846 31651 consensus_meta.cc:287] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.907s user 0.008s sys 0.005s
I20260807 01:48:04.639364 31117 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:04.629650 31458 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:04.677071 31117 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:04.679116 31458 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:04.680333 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:04.680933 31458 raft_consensus.cc:2749] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:04.681713 31458 raft_consensus.cc:2749] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:04.680195 31530 consensus_queue.cc:1048] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:48:04.683032 31530 consensus_queue.cc:1243] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer 57bb4985157447f0a3ab680fa0026f12 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260807 01:48:04.684437 31290 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:04.685104 31290 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:04.686702 28707 pending_rounds.cc:85] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12: Aborting all ops after (but not including) 2
I20260807 01:48:04.687124 28707 pending_rounds.cc:107] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260807 01:48:04.687383 28707 raft_consensus.cc:2889] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 01:48:04.687712 31290 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:04.672734 31568 raft_consensus.cc:2804] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:48:04.695019 31568 raft_consensus.cc:697] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:04.696235 31568 consensus_queue.cc:237] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:04.673100 31492 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:04.699170 31492 raft_consensus.cc:3060] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:04.702899 31632 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:04.704835 28702 raft_consensus.cc:1240] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5. Ops: []
I20260807 01:48:04.629400 31658 raft_consensus.cc:2804] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:04.706529 31658 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:04.706972 31658 raft_consensus.cc:3060] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:04.673478 31484 raft_consensus.cc:515] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:04.723485 28711 raft_consensus.cc:606] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [term 6 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:48:04.739562 27330 raft_consensus.cc:1275] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260807 01:48:04.741415 27330 pending_rounds.cc:85] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade: Aborting all ops after (but not including) 6
I20260807 01:48:04.742092 27330 pending_rounds.cc:107] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260807 01:48:04.744143 27330 raft_consensus.cc:2889] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260807 01:48:04.743014 31477 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s user 0.013s sys 0.031s
W20260807 01:48:04.673825 31650 consensus_meta.cc:287] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade: Time spent flushing consensus metadata: real 0.981s user 0.009s sys 0.003s
I20260807 01:48:04.766268 27326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36604 (request call id 714) took 1140 ms. Trace:
I20260807 01:48:04.782426 27326 rpcz_store.cc:276] 0807 01:48:03.629857 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:03.647753 (+ 17896us) service_pool.cc:224] Handling call
0807 01:48:03.648122 (+ 369us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:03.723064 (+ 74942us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:03.723097 (+ 33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:03.723141 (+ 44us) raft_consensus.cc:1645] Marking committed up to 5
0807 01:48:03.723174 (+ 33us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:04.765595 (+1042421us) spinlock_profiling.cc:243] Waited 74.7 ms on lock 0x7b5400408154. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:04.765718 (+ 123us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:04.766064 (+ 346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":171854080}
I20260807 01:48:04.783241 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:04.785935 27103 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1721) took 1204 ms. Trace:
I20260807 01:48:04.787585 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:04.789587 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:04.756389 31649 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
W20260807 01:48:04.752934 31477 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.695s user 0.014s sys 0.031s
I20260807 01:48:04.793124 27101 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52556 (request call id 614) took 1038 ms. Trace:
I20260807 01:48:04.793484 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:04.786690 27103 rpcz_store.cc:276] 0807 01:48:03.581027 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:03.595930 (+ 14903us) service_pool.cc:224] Handling call
0807 01:48:03.596274 (+ 344us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:03.605134 (+ 8860us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:03.605167 (+ 33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:03.617385 (+ 12218us) log.cc:844] Serialized 26 byte log entry
0807 01:48:03.680557 (+ 63172us) raft_consensus.cc:1645] Marking committed up to 9
0807 01:48:03.680601 (+ 44us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:04.776974 (+1096373us) spinlock_profiling.cc:243] Waited 2.76 ms on lock 0x7b540002dad4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:04.777047 (+ 73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:04.777115 (+ 68us) raft_consensus.cc:1685] finished
0807 01:48:04.777130 (+ 15us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:04.785802 (+ 8672us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7237,"raft.queue_time_us":5815,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":8004,"spinlock_wait_cycles":6408704,"thread_start_us":47159,"threads_started":1,"wal-append.queue_time_us":59120,"wal-append.run_cpu_time_us":1956,"wal-append.run_wall_time_us":1033911}
I20260807 01:48:04.795974 27102 raft_consensus.cc:2368] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6.
I20260807 01:48:04.794914 31650 consensus_queue.cc:1059] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.104s
I20260807 01:48:04.894201 31563 raft_consensus.cc:3060] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
W20260807 01:48:04.857311 27338 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33976 (request call id 2544) took 2024 ms (client timeout 1987 ms). Trace:
I20260807 01:48:04.898679 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:04.797572 31117 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:04.858604 27337 raft_consensus.cc:1275] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260807 01:48:04.878865 31639 raft_consensus.cc:3055] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:48:04.900161 31642 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:04.878664 31484 leader_election.cc:290] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:04.968256 27216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41230 (request call id 491) took 1014 ms. Trace:
I20260807 01:48:04.971452 27216 rpcz_store.cc:276] 0807 01:48:03.953728 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:03.954015 (+ 287us) service_pool.cc:224] Handling call
0807 01:48:03.954376 (+ 361us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:04.680129 (+725753us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:04.680161 (+ 32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:04.680215 (+ 54us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:48:04.680244 (+ 29us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:04.967881 (+287637us) spinlock_profiling.cc:243] Waited 725 ms on lock 0x7b54001dfe54. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:04.967949 (+ 68us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:04.968153 (+ 204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1668622080}
I20260807 01:48:04.919597 27104 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 6
I20260807 01:48:04.873288 31576 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:04.672369 28706 rpcz_store.cc:276] 0807 01:48:02.631111 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:03.055928 (+424817us) service_pool.cc:224] Handling call
0807 01:48:03.082625 (+ 26697us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:03.685559 (+602934us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:04.409645 (+724086us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b5400660dd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:04.409884 (+ 239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1264628608}
I20260807 01:48:04.897013 26952 catalog_manager.cc:5697] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
W20260807 01:48:04.962853 31453 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.905s user 0.029s sys 0.034s
I20260807 01:48:04.868264 27099 raft_consensus.cc:1240] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 6. Current term is 7. Ops: []
I20260807 01:48:05.003650 27099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1718) took 1423 ms. Trace:
I20260807 01:48:05.004360 27099 rpcz_store.cc:276] 0807 01:48:03.579974 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:03.588095 (+ 8121us) service_pool.cc:224] Handling call
0807 01:48:04.867807 (+1279712us) spinlock_profiling.cc:243] Waited 1.93 ms on lock 0x7b5800024c98. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e0521 00007f7d522e03d2 00007f7d5b24c219 00007f7d5a39f640 00007f7d5a467ebb 00007f7d5a467f41 00007f7d5a429dd0 00007f7d5a414739 00007f7d5a4141e2 00007f7d53959d34
0807 01:48:04.868046 (+ 239us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:05.003166 (+135120us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:05.003555 (+ 389us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4428672}
I20260807 01:48:04.856282 28702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
mode: GRACEFUL
new_leader_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:34614
I20260807 01:48:04.799096 27101 rpcz_store.cc:276] 0807 01:48:03.755046 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:04.787105 (+1032059us) service_pool.cc:224] Handling call
0807 01:48:04.793046 (+ 5941us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:05.005836 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:04.871503 31582 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:48:04.853976 31694 log.cc:927] Time spent T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.061s user 0.000s sys 0.000s
I20260807 01:48:04.927887 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:05.029390 27333 raft_consensus.cc:2393] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 7: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:04.928802 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:05.030789 27103 raft_consensus.cc:2468] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:48:05.031368 27103 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52556 (request call id 627) took 1066 ms. Trace:
I20260807 01:48:05.031683 27103 rpcz_store.cc:276] 0807 01:48:03.964827 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:04.928516 (+963689us) service_pool.cc:224] Handling call
0807 01:48:05.031300 (+102784us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:04.915648 27326 raft_consensus.cc:1275] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260807 01:48:05.028136 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:04.931952 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:04.934466 27331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36604 (request call id 713) took 1308 ms. Trace:
I20260807 01:48:04.939527 28714 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:48:05.033702 27327 raft_consensus.cc:2468] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
W20260807 01:48:04.939266 27338 rpcz_store.cc:269] 0807 01:48:02.833131 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:03.308110 (+474979us) service_pool.cc:224] Handling call
0807 01:48:04.857013 (+1548903us) spinlock_profiling.cc:243] Waited 67.8 ms on lock 0x7b54002742d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:04.857160 (+ 147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155972864}
I20260807 01:48:04.983891 27108 raft_consensus.cc:3055] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 01:48:05.035655 27108 raft_consensus.cc:740] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:05.036573 27108 consensus_queue.cc:260] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: false } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:05.037555 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:05.033578 27331 rpcz_store.cc:276] 0807 01:48:03.625789 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:03.626681 (+ 892us) service_pool.cc:224] Handling call
0807 01:48:03.627285 (+ 604us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:03.634927 (+ 7642us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:03.634963 (+ 36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:03.635531 (+ 568us) log.cc:844] Serialized 26 byte log entry
0807 01:48:03.757508 (+121977us) raft_consensus.cc:1645] Marking committed up to 5
0807 01:48:03.757548 (+ 40us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:03.757614 (+ 66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:04.934206 (+1176592us) raft_consensus.cc:1685] finished
0807 01:48:04.934235 (+ 29us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:04.934403 (+ 168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6914,"raft.queue_time_us":6330,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":12923,"spinlock_wait_cycles":8448,"thread_start_us":89480,"threads_started":1,"wal-append.queue_time_us":382968}
W20260807 01:48:04.953296 31161 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s user 0.039s sys 0.028s
I20260807 01:48:04.998421 26952 catalog_manager.cc:5697] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 4 to 5, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 5 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:48:04.859138 31003 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:05.000052 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:05.050246 27100 raft_consensus.cc:2393] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 5: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:04.941071 31639 raft_consensus.cc:740] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:05.041097 26952 catalog_manager.cc:5697] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 3 to 5, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 5 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:48:05.050518 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:05.046228 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:05.033439 27102 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41080 (request call id 3003) took 1062 ms. Trace:
I20260807 01:48:05.051153 27101 raft_consensus.cc:2468] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:48:05.051517 27099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52556 (request call id 622) took 1147 ms. Trace:
I20260807 01:48:05.051689 27101 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41080 (request call id 3008) took 1070 ms. Trace:
I20260807 01:48:05.051985 27099 rpcz_store.cc:276] 0807 01:48:03.903940 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:05.045836 (+1141896us) service_pool.cc:224] Handling call
0807 01:48:05.051446 (+ 5610us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:05.051394 27102 rpcz_store.cc:276] 0807 01:48:03.972039 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:04.933595 (+961556us) service_pool.cc:224] Handling call
0807 01:48:05.033359 (+ 99764us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:05.052227 27101 rpcz_store.cc:276] 0807 01:48:03.981285 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:05.050216 (+1068931us) service_pool.cc:224] Handling call
0807 01:48:05.051621 (+ 1405us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:05.041886 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:05.006595 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 378918us
I20260807 01:48:05.055070 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:48:05.025843 31453 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.968s user 0.030s sys 0.034s
I20260807 01:48:05.066318 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:05.072270 26952 catalog_manager.cc:5697] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 4 to 6, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 6 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } }
I20260807 01:48:05.075453 26952 catalog_manager.cc:5697] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 7 to 8, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 8 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:48:05.079741 27099 raft_consensus.cc:1240] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 4. Current term is 5. Ops: []
I20260807 01:48:05.080593 27099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 625) took 1116 ms. Trace:
I20260807 01:48:05.057538 27103 raft_consensus.cc:2393] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 6: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:05.081043 27099 rpcz_store.cc:276] 0807 01:48:03.964087 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:05.079185 (+1115098us) service_pool.cc:224] Handling call
0807 01:48:05.079539 (+ 354us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:05.080312 (+ 773us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:05.080528 (+ 216us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 01:48:05.049466 31161 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.638s user 0.039s sys 0.028s
W20260807 01:48:05.085777 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:05.067081 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:05.077320 26952 catalog_manager.cc:5697] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: config changed from index 2 to 4, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:48:05.089149 26952 catalog_manager.cc:5697] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 5 to 6, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 6 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:48:05.083562 27099 raft_consensus.cc:1275] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 01:48:05.092331 27099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 626) took 1127 ms. Trace:
I20260807 01:48:05.092803 27099 rpcz_store.cc:276] 0807 01:48:03.964509 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:05.082789 (+1118280us) service_pool.cc:224] Handling call
0807 01:48:05.083191 (+ 402us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:05.092022 (+ 8831us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:05.092266 (+ 244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:05.092557 27101 raft_consensus.cc:1275] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260807 01:48:05.094902 27100 raft_consensus.cc:1240] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 4. Current term is 6. Ops: []
I20260807 01:48:05.095813 27100 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 624) took 1191 ms. Trace:
I20260807 01:48:05.096309 27100 rpcz_store.cc:276] 0807 01:48:03.904622 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:05.067073 (+1162451us) service_pool.cc:224] Handling call
0807 01:48:05.094687 (+ 27614us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:05.095532 (+ 845us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:05.095749 (+ 217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:05.081766 27103 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41080 (request call id 3005) took 1104 ms. Trace:
I20260807 01:48:05.097476 27103 rpcz_store.cc:276] 0807 01:48:03.977526 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:05.054788 (+1077262us) service_pool.cc:224] Handling call
0807 01:48:05.081692 (+ 26904us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:05.099256 27103 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 3006) took 1121 ms. Trace:
I20260807 01:48:05.099745 27103 rpcz_store.cc:276] 0807 01:48:03.977804 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:05.098329 (+1120525us) service_pool.cc:224] Handling call
0807 01:48:05.098712 (+ 383us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:05.098907 (+ 195us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:05.098932 (+ 25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:05.098978 (+ 46us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:48:05.099011 (+ 33us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:05.099065 (+ 54us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:05.099196 (+ 131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:05.101507 27103 raft_consensus.cc:1275] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260807 01:48:05.086448 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:05.049949 28702 raft_consensus.cc:606] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEADER]: Received request to transfer leadership to TS 3478729505b046c2a0051c2d0130873f
I20260807 01:48:05.081250 27104 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:05.126329 27104 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41080 (request call id 3000) took 1160 ms. Trace:
I20260807 01:48:05.127473 27104 rpcz_store.cc:276] 0807 01:48:03.965737 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:04.793135 (+827398us) service_pool.cc:224] Handling call
0807 01:48:05.126252 (+333117us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 01:48:05.109231 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:05.094237 27101 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 3001) took 1127 ms. Trace:
I20260807 01:48:05.150020 27104 raft_consensus.cc:1275] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 01:48:05.132831 27101 rpcz_store.cc:276] 0807 01:48:03.966985 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:05.091813 (+1124828us) service_pool.cc:224] Handling call
0807 01:48:05.092189 (+ 376us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:05.093919 (+ 1730us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:05.094147 (+ 228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:05.065337 31003 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:05.151096 27104 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 3009) took 1169 ms. Trace:
I20260807 01:48:05.117434 31704 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:05.177398 27101 raft_consensus.cc:1275] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260807 01:48:05.123751 27102 raft_consensus.cc:2468] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:48:05.183490 27102 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52556 (request call id 629) took 1218 ms. Trace:
I20260807 01:48:05.184381 27102 rpcz_store.cc:276] 0807 01:48:03.965384 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:05.053430 (+1088046us) service_pool.cc:224] Handling call
0807 01:48:05.183410 (+129980us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:05.117870 26952 catalog_manager.cc:5697] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 6 to 7, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 7 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:48:05.169167 27104 rpcz_store.cc:276] 0807 01:48:03.981576 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:05.128451 (+1146875us) service_pool.cc:224] Handling call
0807 01:48:05.149533 (+ 21082us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:05.150767 (+ 1234us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:05.151028 (+ 261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:05.189550 27103 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 3007) took 1211 ms. Trace:
I20260807 01:48:05.190522 27103 rpcz_store.cc:276] 0807 01:48:03.980910 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:05.100829 (+1119919us) service_pool.cc:224] Handling call
0807 01:48:05.101160 (+ 331us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:05.189270 (+ 88110us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:05.189491 (+ 221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:05.201601 27101 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 3010) took 1219 ms. Trace:
I20260807 01:48:05.202240 27101 rpcz_store.cc:276] 0807 01:48:03.981855 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:05.151842 (+1169987us) service_pool.cc:224] Handling call
0807 01:48:05.177006 (+ 25164us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:05.178203 (+ 1197us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:05.201523 (+ 23320us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:05.188624 27099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 3002) took 1220 ms. Trace:
I20260807 01:48:05.206590 27099 rpcz_store.cc:276] 0807 01:48:03.969495 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:05.093792 (+1124297us) service_pool.cc:224] Handling call
0807 01:48:05.187781 (+ 93989us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:05.187989 (+ 208us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:05.188016 (+ 27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:05.188062 (+ 46us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:48:05.188102 (+ 40us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:05.188155 (+ 53us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:05.188559 (+ 404us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:05.187084 26952 catalog_manager.cc:5697] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 4 to 5, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 5 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:48:05.209702 26952 catalog_manager.cc:5697] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:48:05.215229 27100 raft_consensus.cc:1275] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:48:05.216149 27100 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 3004) took 1242 ms. Trace:
I20260807 01:48:05.216585 27100 rpcz_store.cc:276] 0807 01:48:03.977104 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:05.097224 (+1120120us) service_pool.cc:224] Handling call
0807 01:48:05.185290 (+ 88066us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:05.215846 (+ 30556us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:05.216082 (+ 236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:05.229501 27104 raft_consensus.cc:1275] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
W20260807 01:48:05.244822 28660 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s user 0.023s sys 0.072s
W20260807 01:48:05.245388 28660 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s user 0.023s sys 0.072s
W20260807 01:48:05.245819 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 599034us
W20260807 01:48:05.237816 27287 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s user 0.022s sys 0.098s
I20260807 01:48:05.206583 31654 raft_consensus.cc:493] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:05.248154 27287 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.649s user 0.022s sys 0.098s
I20260807 01:48:05.117210 31639 consensus_queue.cc:260] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:05.251286 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 654221us
I20260807 01:48:05.253820 27287 leader_election.cc:304] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:05.255076 31290 raft_consensus.cc:2749] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:05.256173 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260807 01:48:05.253554 31654 raft_consensus.cc:515] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:05.258358 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:48:05.259096 27287 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:05.259399 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:48:05.260826 27287 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:05.249029 28660 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:05.199946 31704 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:05.253221 27105 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.591s user 0.013s sys 0.012s
W20260807 01:48:05.270718 27105 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.608s user 0.014s sys 0.013s
I20260807 01:48:05.273396 27105 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 619) took 1430 ms. Trace:
I20260807 01:48:05.269632 31290 raft_consensus.cc:2804] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:48:05.268981 31582 raft_consensus.cc:2804] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:48:05.268599 27287 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:05.273926 27105 rpcz_store.cc:276] 0807 01:48:03.843291 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:03.846104 (+ 2813us) service_pool.cc:224] Handling call
0807 01:48:03.846461 (+ 357us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:03.859448 (+ 12987us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:03.902287 (+ 42839us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:03.917669 (+ 15382us) log.cc:844] Serialized 26 byte log entry
0807 01:48:04.597040 (+679371us) raft_consensus.cc:1645] Marking committed up to 7
0807 01:48:05.271182 (+674142us) log.cc:844] Serialized 14 byte log entry
0807 01:48:05.273000 (+ 1818us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:05.273085 (+ 85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:05.273136 (+ 51us) raft_consensus.cc:1685] finished
0807 01:48:05.273148 (+ 12us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:05.273326 (+ 178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":310,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":68986,"thread_start_us":1242026,"threads_started":2,"wal-append.queue_time_us":830419}
W20260807 01:48:05.275365 27110 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 1726) took 1583 ms (client timeout 1564 ms). Trace:
W20260807 01:48:05.312835 27110 rpcz_store.cc:269] 0807 01:48:03.689290 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:03.696918 (+ 7628us) service_pool.cc:224] Handling call
0807 01:48:05.272759 (+1575841us) spinlock_profiling.cc:243] Waited 74.9 ms on lock 0x7b5400357ad4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:05.272867 (+ 108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172282624}
I20260807 01:48:05.275648 27213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41230 (request call id 494) took 1268 ms. Trace:
W20260807 01:48:05.274253 27170 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.603s user 0.003s sys 0.023s
W20260807 01:48:05.277009 27289 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s user 0.006s sys 0.022s
W20260807 01:48:05.316562 27289 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s user 0.006s sys 0.022s
W20260807 01:48:05.316967 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 600947us
I20260807 01:48:05.285157 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.318537 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:05.319182 27338 raft_consensus.cc:2376] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 11. Re-sending same reply.
I20260807 01:48:05.320083 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:05.320746 27338 raft_consensus.cc:2376] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3. Re-sending same reply.
I20260807 01:48:05.314209 27213 rpcz_store.cc:276] 0807 01:48:04.004401 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:04.016767 (+ 12366us) service_pool.cc:224] Handling call
0807 01:48:05.272235 (+1255468us) spinlock_profiling.cc:243] Waited 68 ms on lock 0x7b54000c62d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:05.272386 (+ 151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156508032}
I20260807 01:48:05.321674 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.285450 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.296490 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:48:05.279419 28709 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.565s user 0.006s sys 0.015s
W20260807 01:48:05.323446 28709 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.609s user 0.006s sys 0.015s
I20260807 01:48:05.324010 27338 raft_consensus.cc:2376] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 11. Re-sending same reply.
I20260807 01:48:05.325248 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.325963 27338 raft_consensus.cc:2468] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:48:05.326957 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.327837 27338 raft_consensus.cc:2410] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260807 01:48:05.328709 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.331072 27336 raft_consensus.cc:2393] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 5: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
W20260807 01:48:05.331350 28711 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.602s user 0.054s sys 0.114s
W20260807 01:48:05.351752 28711 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.623s user 0.055s sys 0.114s
W20260807 01:48:05.316179 27170 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.645s user 0.004s sys 0.023s
W20260807 01:48:05.283032 31269 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.746s user 0.002s sys 0.020s
I20260807 01:48:05.337749 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:05.284178 31290 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:05.283872 31582 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:05.332310 31117 raft_consensus.cc:993] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12: : Instructing follower 3478729505b046c2a0051c2d0130873f to start an election
W20260807 01:48:05.340969 28655 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s user 0.013s sys 0.017s
W20260807 01:48:05.405421 28655 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s user 0.013s sys 0.017s
W20260807 01:48:05.405802 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 629552us
I20260807 01:48:05.299742 31484 consensus_queue.cc:1059] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260807 01:48:05.352262 27338 raft_consensus.cc:2410] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260807 01:48:05.289263 31705 raft_consensus.cc:493] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:05.276859 31416 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.868s user 0.018s sys 0.022s
I20260807 01:48:05.400240 27331 raft_consensus.cc:2368] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6.
W20260807 01:48:05.296687 31420 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s user 0.009s sys 0.017s
W20260807 01:48:05.402118 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 768831us
W20260807 01:48:05.329824 28707 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s user 0.012s sys 0.013s
I20260807 01:48:05.337577 31568 raft_consensus.cc:2804] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:48:05.409188 31582 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Advancing to term 8
W20260807 01:48:05.361554 31269 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.830s user 0.002s sys 0.020s
I20260807 01:48:05.404210 31290 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Advancing to term 8
W20260807 01:48:05.415459 31416 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.007s user 0.018s sys 0.023s
W20260807 01:48:05.461758 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 404227us
I20260807 01:48:05.408310 31707 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:48:05.415995 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.361361 31717 raft_consensus.cc:2804] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:48:05.409847 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
W20260807 01:48:05.535756 27172 leader_election.cc:336] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Timed out: RequestConsensusVote RPC to 127.24.4.1:35499 timed out after 1.689s (SENT)
I20260807 01:48:05.534340 27338 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8.
I20260807 01:48:05.537060 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:05.535881 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.527595 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.447674 31161 raft_consensus.cc:1081] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEADER]: Signalling peer 3478729505b046c2a0051c2d0130873f to start an election
W20260807 01:48:05.484117 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:05.516767 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 238959us
W20260807 01:48:05.546617 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:05.468673 31716 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:48:05.529418 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:05.468401 31003 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:05.582602 31716 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:05.583709 31563 consensus_meta.cc:287] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade: Time spent flushing consensus metadata: real 0.643s user 0.013s sys 0.000s
W20260807 01:48:05.467118 28707 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.777s user 0.013s sys 0.014s
I20260807 01:48:05.513363 31726 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:05.531679 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 62725.1us
I20260807 01:48:05.352737 30888 consensus_queue.cc:260] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:05.539978 27336 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:48:05.537667 27338 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:05.442812 31453 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:48:05.466917 31420 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.708s user 0.009s sys 0.018s
I20260807 01:48:05.515693 31642 consensus_queue.cc:1048] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
W20260807 01:48:05.514127 31492 consensus_meta.cc:287] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade: Time spent flushing consensus metadata: real 0.814s user 0.025s sys 0.004s
I20260807 01:48:05.579643 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:48:05.580904 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:05.466955 31568 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:05.529724 27222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41230 (request call id 495) took 1500 ms. Trace:
I20260807 01:48:05.537300 31704 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:05.467569 31720 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:05.470017 31530 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:05.583386 31003 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:05.544973 31707 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:05.445170 31705 raft_consensus.cc:515] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:05.590977 31117 raft_consensus.cc:3055] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Stepping down as leader of term 4
W20260807 01:48:05.602067 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:05.599472 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:05.652638 27336 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
W20260807 01:48:05.647869 27170 leader_election.cc:336] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.896s (SENT)
W20260807 01:48:05.651288 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 204604us
I20260807 01:48:05.650040 27331 raft_consensus.cc:2468] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:48:05.660717 27338 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
W20260807 01:48:05.660830 27170 leader_election.cc:336] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.979s (SENT)
I20260807 01:48:05.663903 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.664584 27331 raft_consensus.cc:2393] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 7: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:05.665253 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:05.650368 27222 rpcz_store.cc:276] 0807 01:48:04.029266 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:04.029396 (+ 130us) service_pool.cc:224] Handling call
0807 01:48:04.582492 (+553096us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:04.582723 (+ 231us) raft_consensus.cc:1534] Early marking committed up to index 7
0807 01:48:04.990046 (+407323us) log.cc:844] Serialized 14 byte log entry
0807 01:48:05.529322 (+539276us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:05.529379 (+ 57us) raft_consensus.cc:1645] Marking committed up to 7
0807 01:48:05.529436 (+ 57us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:05.529528 (+ 92us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:05.529659 (+ 131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43136,"prepare.queue_time_us":499650,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":125694,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":6909,"spinlock_wait_cycles":80875264,"thread_start_us":902564,"threads_started":3,"wal-append.queue_time_us":608947}
W20260807 01:48:05.668818 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:05.584297 31563 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:05.599583 31715 log.cc:927] Time spent T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.248s user 0.000s sys 0.000s
I20260807 01:48:05.688073 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.689929 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
W20260807 01:48:05.602757 31709 log.cc:927] Time spent T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.141s user 0.000s sys 0.000s
I20260807 01:48:05.651818 30888 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:05.659317 31568 raft_consensus.cc:3060] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:05.620908 31654 leader_election.cc:290] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:05.696565 27338 raft_consensus.cc:2410] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260807 01:48:05.717077 28660 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.4.1:35499 timed out after 1.553s (SENT) [suppressed 50 similar messages]
I20260807 01:48:05.592274 31203 consensus_queue.cc:1048] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.100s
I20260807 01:48:05.720883 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:05.638036 31509 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:05.700496 31642 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:05.599478 31754 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:05.666690 31003 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:05.659736 31117 raft_consensus.cc:740] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:05.666260 31492 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:05.693179 31631 consensus_queue.cc:1048] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260807 01:48:05.726289 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:05.760596 27338 raft_consensus.cc:3060] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:05.658875 31716 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:05.710350 31658 consensus_meta.cc:287] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade: Time spent flushing consensus metadata: real 1.003s user 0.018s sys 0.001s
I20260807 01:48:05.719228 27336 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:05.751536 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:05.753165 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:05.754820 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:05.757131 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:05.711632 31704 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:05.724690 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.665799 31530 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:05.670470 31717 raft_consensus.cc:697] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:05.666117 31453 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:05.758829 27102 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 3016) took 1079 ms. Trace:
I20260807 01:48:05.783185 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:05.787488 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:05.666069 31720 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:05.763128 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:05.791811 27099 raft_consensus.cc:2468] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:48:05.693764 31563 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:05.783676 31704 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:05.793071 31753 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:05.793628 31753 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:05.794651 31716 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:05.795221 31716 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:05.796106 31753 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:05.603142 31726 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:48:05.731441 31642 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:05.766556 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:05.773869 27104 raft_consensus.cc:2410] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260807 01:48:05.774233 27105 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 7.
I20260807 01:48:05.808845 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:05.772949 27110 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:48:05.809759 27331 raft_consensus.cc:2410] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260807 01:48:05.810196 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:05.811023 27110 raft_consensus.cc:2410] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260807 01:48:05.765463 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:05.811859 27212 raft_consensus.cc:2410] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260807 01:48:05.811038 27287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:05.812554 27287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:05.812860 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:48:05.813130 27287 connection.cc:256] RPC call timeout handler was delayed by -1.1963795938063413s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1564s for a response
W20260807 01:48:05.813560 27287 connection.cc:256] RPC call timeout handler was delayed by -1.1954795939382166s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1565s for a response
I20260807 01:48:05.780550 27214 raft_consensus.cc:1240] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260807 01:48:05.783001 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:05.785810 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.789011 27338 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:48:05.762426 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:05.785825 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:05.786939 27229 raft_consensus.cc:2410] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260807 01:48:05.786384 27102 rpcz_store.cc:276] 0807 01:48:04.678672 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:05.206976 (+528304us) service_pool.cc:224] Handling call
0807 01:48:05.246626 (+ 39650us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:05.246855 (+ 229us) raft_consensus.cc:1534] Early marking committed up to index 8
0807 01:48:05.484392 (+237537us) log.cc:844] Serialized 14 byte log entry
0807 01:48:05.722503 (+238111us) log.cc:844] Serialized 14 byte log entry
0807 01:48:05.757742 (+ 35239us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:05.757797 (+ 55us) raft_consensus.cc:1645] Marking committed up to 8
0807 01:48:05.758256 (+ 459us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:05.758354 (+ 98us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:05.758486 (+ 132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16825,"prepare.queue_time_us":353493,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":101,"raft.queue_time_us":78785,"thread_start_us":492578,"threads_started":3}
I20260807 01:48:05.765738 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:05.779407 31756 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:05.802974 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:05.804147 31704 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:05.797364 31716 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:05.794032 31563 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:05.806525 31726 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:05.732470 31754 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:05.778792 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:05.800254 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:48:05.813881 27287 connection.cc:256] RPC call timeout handler was delayed by -1.1405795938335359s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1626s for a response
W20260807 01:48:05.847239 27287 connection.cc:256] RPC call timeout handler was delayed by -1.0136795938014984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1767s for a response
I20260807 01:48:05.826900 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.827657 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:05.848155 27214 raft_consensus.cc:3060] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:05.825789 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:05.848938 27170 leader_election.cc:336] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 2.547s (SENT)
I20260807 01:48:05.848600 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.828354 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:05.849751 27170 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:05.849900 27338 raft_consensus.cc:2376] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6. Re-sending same reply.
I20260807 01:48:05.850330 27218 raft_consensus.cc:2468] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:48:05.828372 27216 raft_consensus.cc:2468] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:48:05.814229 27222 raft_consensus.cc:2468] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 2.
I20260807 01:48:05.852445 27103 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 3018) took 1173 ms. Trace:
I20260807 01:48:05.851096 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.787990 31416 raft_consensus.cc:493] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:05.854306 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:05.852850 27103 rpcz_store.cc:276] 0807 01:48:04.679237 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:05.207057 (+527820us) service_pool.cc:224] Handling call
0807 01:48:05.207450 (+ 393us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:05.207684 (+ 234us) raft_consensus.cc:1534] Early marking committed up to index 6
0807 01:48:05.464821 (+257137us) log.cc:844] Serialized 14 byte log entry
0807 01:48:05.851004 (+386183us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:05.851069 (+ 65us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:48:05.851125 (+ 56us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:05.851204 (+ 79us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:05.852382 (+ 1178us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":334920,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":4746,"raft.queue_time_us":140465,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":1551,"thread_start_us":523988,"threads_started":3}
I20260807 01:48:05.832032 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:05.840127 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.848536 27287 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:05.784766 31117 consensus_queue.cc:260] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:05.856503 31117 raft_consensus.cc:3060] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:05.856894 31705 leader_election.cc:290] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:05.858217 31726 raft_consensus.cc:2749] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 01:48:05.859084 31726 consensus_queue.cc:1059] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 01:48:05.855535 27328 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8.
I20260807 01:48:05.849637 27100 raft_consensus.cc:2468] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 7.
I20260807 01:48:05.852952 28660 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:05.874207 31306 raft_consensus.cc:2804] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:05.874409 31003 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:05.874831 31306 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:05.875366 31306 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:05.877427 31290 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:05.880069 31290 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:05.879705 31717 consensus_queue.cc:237] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:05.832409 31756 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:05.853266 27338 raft_consensus.cc:2468] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:48:05.830282 27101 raft_consensus.cc:1275] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:48:05.829252 31758 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:05.884037 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.884138 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.884948 27336 raft_consensus.cc:2468] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8.
I20260807 01:48:05.885259 27338 raft_consensus.cc:2393] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 7: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:05.854525 27218 raft_consensus.cc:1275] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260807 01:48:05.780210 31658 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:05.860304 27287 leader_election.cc:400] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 election: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 5. Current term is 6.
I20260807 01:48:05.895164 27287 leader_election.cc:403] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260807 01:48:05.896030 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.895399 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:05.897310 27328 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:48:05.897749 27229 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:41230
I20260807 01:48:05.898443 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.899159 27328 raft_consensus.cc:2468] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
W20260807 01:48:05.900094 31683 log.cc:927] Time spent T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 1.996s user 0.011s sys 0.010s
I20260807 01:48:05.901829 31658 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:05.919019 27214 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:48:05.898427 27229 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:48:05.943629 27229 raft_consensus.cc:3060] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:48:05.906792 28707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48350 (request call id 851) took 1221 ms. Trace:
I20260807 01:48:05.869851 31416 raft_consensus.cc:515] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:05.909425 27330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36604 (request call id 711) took 2284 ms. Trace:
I20260807 01:48:05.908587 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.900028 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.898490 27338 raft_consensus.cc:2376] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7. Re-sending same reply.
W20260807 01:48:05.921520 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 61377.8us
I20260807 01:48:05.916605 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:05.948196 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.948825 27331 raft_consensus.cc:2368] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6.
I20260807 01:48:05.949333 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:48:05.922989 31762 log.cc:927] Time spent T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.063s user 0.000s sys 0.000s
I20260807 01:48:05.894088 31707 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:05.923225 27325 raft_consensus.cc:1275] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260807 01:48:05.905805 31582 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:05.851751 31568 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:05.944265 31203 raft_consensus.cc:2749] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 01:48:05.924732 31492 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:05.929508 31639 raft_consensus.cc:3060] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:05.956008 31568 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:05.934255 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:05.935045 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:05.960104 31416 leader_election.cc:290] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:05.945858 28707 rpcz_store.cc:276] 0807 01:48:04.685647 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:04.685961 (+ 314us) service_pool.cc:224] Handling call
0807 01:48:04.686439 (+ 478us) raft_consensus.cc:1471] Updating replica for 2 ops
0807 01:48:04.688827 (+ 2388us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:04.688863 (+ 36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0807 01:48:04.689653 (+ 790us) log.cc:844] Serialized 52 byte log entry
0807 01:48:05.860518 (+1170865us) spinlock_profiling.cc:243] Waited 4.28 ms on lock 0x7b1800330820. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d5b233f31 00007f7d5b2406d1 00007f7d54fc56ac 00007f7d522fae3b 00007f7d5b1a2fc9 00007f7d52316fdb 00007f7d523152db 00007f7d52317542 00007f7d54fb73fb 00007f7d54fbcf13 00007f7d54fbd2c1
0807 01:48:05.893353 (+ 32835us) raft_consensus.cc:1645] Marking committed up to 2
0807 01:48:05.906210 (+ 12857us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:05.906351 (+ 141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:05.906416 (+ 65us) raft_consensus.cc:1685] finished
0807 01:48:05.906436 (+ 20us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:05.906711 (+ 275us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23793,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":103,"spinlock_wait_cycles":9834752,"thread_start_us":901347,"threads_started":1,"wal-append.queue_time_us":891261}
I20260807 01:48:05.960850 27230 raft_consensus.cc:2468] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:48:05.944885 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.922363 31003 raft_consensus.cc:2749] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 5. Current term is 6.
I20260807 01:48:05.949606 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.950197 27327 raft_consensus.cc:2393] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 6: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:05.964571 27336 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:48:05.966495 27327 raft_consensus.cc:1275] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 11 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260807 01:48:05.946297 27330 rpcz_store.cc:276] 0807 01:48:03.625075 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:03.627867 (+ 2792us) service_pool.cc:224] Handling call
0807 01:48:03.628148 (+ 281us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:04.745366 (+1117218us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:05.909172 (+1163806us) spinlock_profiling.cc:243] Waited 911 ms on lock 0x7b540012f2d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:05.909362 (+ 190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2095940992}
I20260807 01:48:05.951560 27287 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:05.896946 31758 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:05.968951 27333 raft_consensus.cc:2468] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:48:05.951840 27338 raft_consensus.cc:2468] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:48:05.970237 27333 raft_consensus.cc:3055] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:48:05.970729 27333 raft_consensus.cc:740] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:05.954643 31726 raft_consensus.cc:493] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:05.970392 27229 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:05.971817 27327 raft_consensus.cc:1275] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 01:48:05.971473 27287 leader_election.cc:400] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 6. Current term is 7.
W20260807 01:48:05.972260 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:05.972560 27287 leader_election.cc:403] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260807 01:48:05.970968 27338 raft_consensus.cc:1275] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260807 01:48:05.974298 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:05.978128 27229 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:05.979719 27330 raft_consensus.cc:1275] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260807 01:48:05.978130 31726 raft_consensus.cc:515] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:05.981058 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:05.974937 31568 raft_consensus.cc:2804] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:48:05.982924 31568 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:05.983331 31568 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:05.985572 31492 raft_consensus.cc:2749] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 6. Current term is 7.
I20260807 01:48:05.988615 27336 raft_consensus.cc:1275] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:48:05.971642 27333 consensus_queue.cc:260] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:05.990731 27287 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:05.995296 31749 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:05.945546 31306 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:05.995396 31639 raft_consensus.cc:2749] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:05.997051 31758 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:05.997514 27333 raft_consensus.cc:3060] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:05.999131 27108 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:05.997534 31306 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:06.005445 31720 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:06.008347 28655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:06.011229 27229 raft_consensus.cc:1240] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 3. Current term is 4. Ops: []
I20260807 01:48:06.012600 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:06.019860 27333 raft_consensus.cc:1275] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 2. (term mismatch)
I20260807 01:48:06.020458 27333 pending_rounds.cc:85] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade: Aborting all ops after (but not including) 1
I20260807 01:48:06.020777 27333 pending_rounds.cc:107] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260807 01:48:06.021181 27333 raft_consensus.cc:2889] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 01:48:06.020586 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:06.025759 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:06.027995 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:06.025501 27214 raft_consensus.cc:2468] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:48:06.028716 27222 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
I20260807 01:48:06.013612 31290 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:05.999185 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:06.030915 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:06.034497 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:06.036324 31754 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:06.038619 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:05.996021 31749 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:06.043325 31642 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:06.027451 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:06.066108 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
W20260807 01:48:06.052847 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 62219.2us
I20260807 01:48:06.068511 27229 raft_consensus.cc:3060] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 6
I20260807 01:48:06.068751 27287 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:06.068892 27064 leader_election.cc:304] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:06.070478 27287 leader_election.cc:304] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
W20260807 01:48:06.055271 27170 leader_election.cc:336] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 3.020s (SENT)
W20260807 01:48:06.058817 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:06.063038 27108 raft_consensus.cc:2410] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260807 01:48:06.060560 31290 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:06.070438 31652 raft_consensus.cc:2749] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:06.073205 31650 raft_consensus.cc:2804] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:06.073489 28707 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
from {username='slave'} at 127.0.0.1:52322
I20260807 01:48:06.074254 28707 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:48:06.065927 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:06.074645 28707 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:06.074865 31749 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:06.075254 27218 raft_consensus.cc:2393] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 6: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:06.075719 28706 raft_consensus.cc:2410] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260807 01:48:06.065198 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:06.072930 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:06.076725 27337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33976 (request call id 2549) took 3147 ms. Trace:
I20260807 01:48:06.076905 27227 raft_consensus.cc:2468] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:48:06.074653 31582 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:06.078898 31758 raft_consensus.cc:2749] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:06.052544 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:06.076139 28709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48350 (request call id 849) took 1461 ms. Trace:
I20260807 01:48:06.082211 28709 rpcz_store.cc:276] 0807 01:48:04.614327 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:04.614508 (+ 181us) service_pool.cc:224] Handling call
0807 01:48:04.614904 (+ 396us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:04.714134 (+ 99230us) raft_consensus.cc:1534] Early marking committed up to index 6
0807 01:48:05.331458 (+617324us) log.cc:844] Serialized 14 byte log entry
0807 01:48:05.581665 (+250207us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:05.581720 (+ 55us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:48:05.581768 (+ 48us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:06.075906 (+494138us) spinlock_profiling.cc:243] Waited 99 ms on lock 0x7b540076bbd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:06.075956 (+ 50us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:06.076083 (+ 127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28841,"prepare.queue_time_us":756564,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":80,"raft.queue_time_us":90670,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":124,"spinlock_wait_cycles":227605888,"thread_start_us":794707,"threads_started":2,"wal-append.queue_time_us":269621}
I20260807 01:48:06.066943 27230 raft_consensus.cc:1240] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 6. Current term is 7. Ops: []
I20260807 01:48:06.073622 31650 raft_consensus.cc:493] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:48:06.085291 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:06.085990 27337 rpcz_store.cc:276] 0807 01:48:02.929282 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:03.514616 (+585334us) service_pool.cc:224] Handling call
0807 01:48:04.858239 (+1343623us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:04.939600 (+ 81361us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:06.076529 (+1136929us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54002742d0. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541dc8d 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:06.076665 (+ 136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3082346240}
I20260807 01:48:06.087133 27222 raft_consensus.cc:1240] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 6. Current term is 7. Ops: []
I20260807 01:48:06.089269 27229 raft_consensus.cc:2468] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
W20260807 01:48:06.090705 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:06.091523 27214 raft_consensus.cc:1240] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 4. Current term is 6. Ops: []
I20260807 01:48:06.087256 31650 raft_consensus.cc:3060] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:06.070254 27064 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:06.102857 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:06.103085 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:06.103621 27230 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:06.094133 27172 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:06.109619 27325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36604 (request call id 730) took 1154 ms. Trace:
I20260807 01:48:06.110059 27325 rpcz_store.cc:276] 0807 01:48:04.955498 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:04.955677 (+ 179us) service_pool.cc:224] Handling call
0807 01:48:05.000927 (+ 45250us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:05.952963 (+952036us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:06.109362 (+156399us) spinlock_profiling.cc:243] Waited 922 ms on lock 0x7b5400166054. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:06.109552 (+ 190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2120278784}
I20260807 01:48:06.106891 27064 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:06.114600 27230 raft_consensus.cc:2468] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
I20260807 01:48:06.112390 27287 leader_election.cc:304] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:06.117614 27172 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:06.118288 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:06.107828 31764 raft_consensus.cc:2804] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:48:06.116149 31642 raft_consensus.cc:2749] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:48:06.119740 31764 raft_consensus.cc:697] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:06.120728 31764 consensus_queue.cc:237] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:06.122826 27287 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:06.123114 31729 consensus_queue.cc:1048] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 01:48:06.124135 27287 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:06.125312 27287 leader_election.cc:304] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:06.129788 31743 raft_consensus.cc:2804] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:48:06.127835 27287 leader_election.cc:304] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:06.126272 27064 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:06.129277 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:06.127969 28660 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:06.129498 27230 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
I20260807 01:48:06.095548 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:06.137905 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.131152 31492 raft_consensus.cc:2804] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:48:06.094726 31726 leader_election.cc:290] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:06.094664 31453 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:06.138703 27337 raft_consensus.cc:2468] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:48:06.138988 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.145442 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.144942 28660 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:06.139824 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.147323 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.140362 31707 consensus_queue.cc:1048] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260807 01:48:06.141481 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 114900us
W20260807 01:48:06.153257 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:06.141402 31662 consensus_queue.cc:1048] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260807 01:48:06.126518 31509 raft_consensus.cc:2804] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:06.147042 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.158603 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.151067 31631 consensus_queue.cc:1059] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260807 01:48:06.140915 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.141637 31770 raft_consensus.cc:2804] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:48:06.151275 27328 raft_consensus.cc:2368] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 3. Current term is 4.
I20260807 01:48:06.149123 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.143879 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.138607 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.153672 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.133055 31003 raft_consensus.cc:2804] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:06.130442 31758 raft_consensus.cc:2749] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:06.140519 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.181490 27336 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
I20260807 01:48:06.134744 31743 raft_consensus.cc:493] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:06.096840 31568 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:06.145146 31639 raft_consensus.cc:2804] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:48:06.138576 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:06.161019 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.186532 27333 raft_consensus.cc:2468] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 3.
I20260807 01:48:06.163301 27325 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 11.
I20260807 01:48:06.148406 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:06.163938 31604 consensus_queue.cc:1059] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 01:48:06.156345 31306 raft_consensus.cc:2804] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:48:06.153918 31453 consensus_queue.cc:1048] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260807 01:48:06.203137 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.203140 31306 raft_consensus.cc:697] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 7 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:06.204941 28707 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:06.205179 31306 consensus_queue.cc:237] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:06.144868 31492 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:06.202667 31716 consensus_queue.cc:1059] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260807 01:48:06.206921 31492 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:06.207722 31716 raft_consensus.cc:3055] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:48:06.208109 31716 raft_consensus.cc:740] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
W20260807 01:48:06.166509 31563 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260807 01:48:06.209236 31716 consensus_queue.cc:260] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:06.210230 31530 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:06.136811 31416 consensus_queue.cc:1048] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.027s
I20260807 01:48:06.211315 31568 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:06.168057 31776 consensus_queue.cc:1048] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260807 01:48:06.168339 31290 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:06.163028 31771 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:06.169041 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:06.175113 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:06.175350 31752 consensus_queue.cc:1048] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 01:48:06.176870 31117 consensus_queue.cc:1059] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:48:06.162485 31509 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:06.216660 31756 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:06.157401 31642 consensus_queue.cc:1048] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
W20260807 01:48:06.217994 27108 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52556 (request call id 628) took 2252 ms (client timeout 1618 ms). Trace:
I20260807 01:48:06.217100 31509 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
W20260807 01:48:06.218518 27108 rpcz_store.cc:269] 0807 01:48:03.965103 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:04.782749 (+817646us) service_pool.cc:224] Handling call
0807 01:48:05.998809 (+1216060us) spinlock_profiling.cc:243] Waited 760 ms on lock 0x7b58005d02d0. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5537a48a 00007f7d553fc827 00007f7d554153c7 00007f7d55416bbe 00007f7d5542265c 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c
0807 01:48:06.217749 (+218940us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54003183d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:06.217908 (+ 159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2206101632}
I20260807 01:48:06.146234 31729 raft_consensus.cc:2804] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:48:06.170635 27331 raft_consensus.cc:2376] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6. Re-sending same reply.
I20260807 01:48:06.220207 31729 raft_consensus.cc:697] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 6 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:06.221320 31729 consensus_queue.cc:237] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:06.220196 31642 raft_consensus.cc:493] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:06.222841 31716 raft_consensus.cc:3060] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:06.223240 31642 raft_consensus.cc:515] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:06.220057 31650 raft_consensus.cc:515] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:06.175192 31631 raft_consensus.cc:3055] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:48:06.179062 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:06.180154 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:06.137243 31720 consensus_queue.cc:1048] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.028s
I20260807 01:48:06.180356 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:06.180899 31003 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:06.144424 31717 consensus_queue.cc:1048] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260807 01:48:06.185689 31639 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:06.183184 31743 raft_consensus.cc:3060] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:06.177637 31754 raft_consensus.cc:2749] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 01:48:06.202422 31604 raft_consensus.cc:3055] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:48:06.177860 28660 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:06.199225 27064 leader_election.cc:304] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
W20260807 01:48:06.247848 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 48757.7us
I20260807 01:48:06.234629 28702 raft_consensus.cc:3055] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 01:48:06.227380 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:06.249404 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.249640 31632 raft_consensus.cc:2804] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:48:06.250234 31632 raft_consensus.cc:493] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:06.250586 28707 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:06.247061 31604 raft_consensus.cc:740] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:06.231297 31631 raft_consensus.cc:740] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:06.227368 31770 raft_consensus.cc:697] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:06.252199 31604 consensus_queue.cc:260] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:06.253105 31770 consensus_queue.cc:237] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:06.243132 31639 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:06.227905 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:06.245190 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:06.260043 28715 raft_consensus.cc:2410] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260807 01:48:06.246212 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:06.261341 28714 raft_consensus.cc:2410] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260807 01:48:06.246352 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.250702 31632 raft_consensus.cc:3060] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:06.263386 31650 leader_election.cc:290] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:06.252426 31631 consensus_queue.cc:260] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:06.265254 31631 raft_consensus.cc:3060] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 6
I20260807 01:48:06.213527 31771 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:06.248555 27064 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:06.253762 31604 raft_consensus.cc:3060] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:06.235119 31736 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:06.266776 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 88996.1us
I20260807 01:48:06.266412 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.277383 27214 raft_consensus.cc:1240] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 2. Current term is 3. Ops: []
I20260807 01:48:06.277369 27338 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 11.
I20260807 01:48:06.238484 27063 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:06.234604 31642 leader_election.cc:290] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:06.243484 27325 raft_consensus.cc:2393] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 6: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:06.233420 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:06.268719 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.269269 31752 raft_consensus.cc:2804] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:48:06.284952 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:06.272815 27064 leader_election.cc:304] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:06.285602 27227 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:48:06.286891 27064 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:06.287050 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:06.287601 27227 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:06.284845 27328 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
I20260807 01:48:06.273447 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:06.269918 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.289397 27333 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
W20260807 01:48:06.290020 27170 leader_election.cc:336] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.953s (SENT)
I20260807 01:48:06.240861 31003 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:06.284655 27331 raft_consensus.cc:2376] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6. Re-sending same reply.
I20260807 01:48:06.290943 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:06.296037 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:06.300305 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:06.302289 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:06.303138 27103 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 10.
I20260807 01:48:06.311450 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:06.314867 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:06.315253 27331 raft_consensus.cc:1240] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 6. Current term is 7. Ops: []
I20260807 01:48:06.316259 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:06.317194 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:06.265815 28702 raft_consensus.cc:740] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:06.343024 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:06.372531 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:06.373662 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:06.311604 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:06.375047 28714 raft_consensus.cc:2410] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260807 01:48:06.328310 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:06.376550 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:06.328037 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:06.343232 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:06.327947 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:06.378240 27229 raft_consensus.cc:2468] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:48:06.378386 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:06.346676 27172 leader_election.cc:336] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Timed out: RequestConsensusVote RPC to 127.24.4.1:35499 timed out after 1.546s (SENT)
I20260807 01:48:06.380100 27172 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:06.282670 31651 raft_consensus.cc:2804] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:48:06.372834 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:06.327517 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:06.329931 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:06.321410 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:48:06.381102 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 91638.1us
I20260807 01:48:06.291222 31752 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:06.391809 27170 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:48:06.381912 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:06.321887 27222 raft_consensus.cc:2410] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260807 01:48:06.371008 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 78958.6us
I20260807 01:48:06.318725 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:06.384488 27100 raft_consensus.cc:2468] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:48:06.283303 31568 raft_consensus.cc:493] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:06.400678 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.401752 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.401785 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.397750 27227 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 8.
I20260807 01:48:06.304533 31652 raft_consensus.cc:2804] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:48:06.409250 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.410192 27337 raft_consensus.cc:2393] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 4: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:06.411912 27170 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:06.412902 27170 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:48:06.309448 31714 raft_consensus.cc:2749] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:06.415217 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:06.314437 31654 raft_consensus.cc:2804] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:48:06.415817 27336 raft_consensus.cc:3060] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 2 FOLLOWER]: Advancing to term 3
I20260807 01:48:06.416662 27333 raft_consensus.cc:2393] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 6: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
W20260807 01:48:06.416666 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 70047.9us
I20260807 01:48:06.417773 27170 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:06.419178 28660 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:48:06.431077 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:06.433477 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:06.434137 27338 raft_consensus.cc:2376] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6. Re-sending same reply.
I20260807 01:48:06.435125 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.435906 27338 raft_consensus.cc:2393] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 7: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:06.442683 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:06.444046 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.444470 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.383383 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:06.447692 28660 leader_election.cc:304] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:06.432700 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.450023 27337 raft_consensus.cc:2468] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:48:06.452376 27330 raft_consensus.cc:2393] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 8: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:06.455418 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.456084 27331 raft_consensus.cc:2468] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:06.387985 31651 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:06.309623 31484 consensus_queue.cc:1059] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.112s
I20260807 01:48:06.406213 31752 raft_consensus.cc:3060] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:06.386494 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:06.455138 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.462042 27336 raft_consensus.cc:2468] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:48:06.462787 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.463524 27330 raft_consensus.cc:2393] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 4: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:06.463677 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.464360 27331 raft_consensus.cc:2468] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
W20260807 01:48:06.470332 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:06.419116 31568 raft_consensus.cc:515] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:06.457679 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.473475 31568 leader_election.cc:290] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:06.413823 31652 raft_consensus.cc:493] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:06.474224 31652 raft_consensus.cc:3060] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:06.416224 31416 raft_consensus.cc:2749] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:06.427335 31453 raft_consensus.cc:2804] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 01:48:06.458953 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.304899 31736 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:06.474261 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:06.472991 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:06.476739 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:06.484586 28709 raft_consensus.cc:2410] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260807 01:48:06.444861 31203 raft_consensus.cc:2804] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:48:06.457958 31654 raft_consensus.cc:697] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:06.482319 28708 raft_consensus.cc:1240] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 7 LEADER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 6. Current term is 7. Ops: []
I20260807 01:48:06.482254 27338 raft_consensus.cc:2393] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 6: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:06.488979 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.455134 31117 raft_consensus.cc:2749] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:48:06.485877 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:06.461915 31651 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:06.496393 28714 raft_consensus.cc:2410] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260807 01:48:06.460117 31776 raft_consensus.cc:2804] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260807 01:48:06.501411 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:06.481777 31453 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:06.505759 28655 leader_election.cc:304] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:06.506686 28714 raft_consensus.cc:1240] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 6. Current term is 8. Ops: []
I20260807 01:48:06.486812 31203 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:06.405606 31492 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:06.446270 31768 raft_consensus.cc:2804] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:06.517655 31768 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:06.518374 31768 raft_consensus.cc:3060] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:06.519811 31639 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:06.520836 31492 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:06.508144 31453 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:06.500098 31776 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 10 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:48:06.526438 31776 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 10 FOLLOWER]: Advancing to term 11
I20260807 01:48:06.530177 31654 consensus_queue.cc:237] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:06.536639 31639 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:06.486323 31743 raft_consensus.cc:515] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:06.537508 31651 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:06.544064 31736 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:06.505100 31742 consensus_queue.cc:1048] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260807 01:48:06.546284 31771 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:06.550045 31743 leader_election.cc:290] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:48:06.382227 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 144537us
I20260807 01:48:06.594841 27063 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:06.597484 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:06.623376 27104 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:06.626971 27287 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:06.632797 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:48:06.599637 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:06.599538 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:06.602214 27067 leader_election.cc:304] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:06.394616 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:06.603006 27110 raft_consensus.cc:2468] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:48:06.598995 27101 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:48:06.603533 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:48:06.609745 27064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:06.616567 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 198749us
I20260807 01:48:06.646706 27214 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 7.
I20260807 01:48:06.650254 27170 leader_election.cc:304] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:06.615891 31809 consensus_queue.cc:1048] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:48:06.651394 27327 raft_consensus.cc:2393] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 4: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:06.474483 27337 raft_consensus.cc:2468] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:48:06.489802 27331 raft_consensus.cc:2468] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:48:06.507977 28710 pending_rounds.cc:85] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12: Aborting all ops after (but not including) 7
I20260807 01:48:06.685278 31764 consensus_queue.cc:1059] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:48:06.490289 28709 raft_consensus.cc:1240] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 7. Current term is 8. Ops: []
I20260807 01:48:06.687299 31764 consensus_queue.cc:1048] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:48:06.688503 27326 raft_consensus.cc:2393] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 8: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:06.641983 26948 catalog_manager.cc:5697] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:48:06.647352 27287 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:06.674896 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.682888 31492 raft_consensus.cc:2749] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260807 01:48:06.683845 31736 consensus_queue.cc:1048] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:48:06.685688 28710 pending_rounds.cc:107] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260807 01:48:06.665899 27330 raft_consensus.cc:1240] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 5. Current term is 6. Ops: []
W20260807 01:48:06.699280 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 49346.9us
I20260807 01:48:06.696301 28710 raft_consensus.cc:2889] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260807 01:48:06.705147 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 75256.1us
I20260807 01:48:06.706878 31743 raft_consensus.cc:2749] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 01:48:06.709075 27287 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:06.708825 28702 consensus_queue.cc:260] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:06.705811 31662 raft_consensus.cc:2804] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:06.710258 31743 raft_consensus.cc:2749] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 01:48:06.712284 31003 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:06.713992 27338 raft_consensus.cc:1275] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 01:48:06.714720 31396 raft_consensus.cc:2804] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260807 01:48:06.715178 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 209389us
I20260807 01:48:06.717603 26952 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46134 (request call id 416) took 3772 ms. Trace:
I20260807 01:48:06.684521 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:48:06.728389 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:06.727671 26948 catalog_manager.cc:5697] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 2 to 5, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4). New cstate: current_term: 5 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:48:06.729432 27327 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:06.513545 31203 raft_consensus.cc:3060] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:06.732946 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:06.725679 31396 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:06.740662 31396 raft_consensus.cc:3060] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:06.740952 31770 raft_consensus.cc:2749] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:06.685560 31716 consensus_queue.cc:1048] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
W20260807 01:48:06.749025 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 147378us
I20260807 01:48:06.749691 28702 raft_consensus.cc:3060] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 2 FOLLOWER]: Advancing to term 3
W20260807 01:48:06.750689 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 156381us
I20260807 01:48:06.751879 27330 raft_consensus.cc:1240] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 3. Current term is 4. Ops: []
I20260807 01:48:06.752161 27063 leader_election.cc:304] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:06.754398 31416 raft_consensus.cc:2749] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 01:48:06.755424 31771 raft_consensus.cc:2749] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:06.724783 26952 rpcz_store.cc:276] 0807 01:48:02.944691 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:02.944908 (+ 217us) service_pool.cc:224] Handling call
0807 01:48:06.717297 (+3772389us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 01:48:05.779901 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 01:48:05.923493 (+143592us) write_op.cc:432] Acquiring schema lock in shared mode
0807 01:48:05.923533 (+ 40us) write_op.cc:435] Acquired schema lock
0807 01:48:05.923550 (+ 17us) tablet.cc:662] Decoding operations
0807 01:48:05.986574 (+ 63024us) write_op.cc:620] Acquiring the partition lock for write op
0807 01:48:05.986647 (+ 73us) write_op.cc:641] Partition lock acquired for write op
0807 01:48:05.986673 (+ 26us) tablet.cc:685] Acquiring locks for 10 operations
0807 01:48:05.987513 (+ 840us) tablet.cc:701] Row locks acquired
0807 01:48:05.987529 (+ 16us) write_op.cc:260] PREPARE: finished
0807 01:48:05.987700 (+ 171us) write_op.cc:270] Start()
0807 01:48:05.987825 (+ 125us) write_op.cc:276] Timestamp: P: 1786067285987642 usec, L: 0
0807 01:48:05.987841 (+ 16us) op_driver.cc:348] REPLICATION: starting
0807 01:48:05.988505 (+ 664us) log.cc:844] Serialized 3984 byte log entry
0807 01:48:06.164589 (+176084us) op_driver.cc:464] REPLICATION: finished
0807 01:48:06.201264 (+ 36675us) write_op.cc:301] APPLY: starting
0807 01:48:06.201335 (+ 71us) tablet.cc:1367] starting BulkCheckPresence
0807 01:48:06.228247 (+ 26912us) tablet.cc:1370] finished BulkCheckPresence
0807 01:48:06.233620 (+ 5373us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 01:48:06.508028 (+274408us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 01:48:06.533395 (+ 25367us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0807 01:48:06.533467 (+ 72us) write_op.cc:312] APPLY: finished
0807 01:48:06.533957 (+ 490us) log.cc:844] Serialized 77 byte log entry
0807 01:48:06.534283 (+ 326us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:48:06.534442 (+ 159us) write_op.cc:454] Released schema lock
0807 01:48:06.534504 (+ 62us) write_op.cc:341] FINISH: Updating metrics
0807 01:48:06.535168 (+ 664us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:48:06.535210 (+ 42us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":36420,"apply.run_cpu_time_us":6478,"apply.run_wall_time_us":333803,"num_ops":10,"prepare.queue_time_us":322315,"prepare.run_cpu_time_us":11586,"prepare.run_wall_time_us":270362,"raft.queue_time_us":98849,"raft.run_cpu_time_us":12638,"raft.run_wall_time_us":36678,"replication_time_us":176595,"thread_start_us":298677,"threads_started":4,"wal-append.queue_time_us":29419}]]}
I20260807 01:48:06.763780 27330 raft_consensus.cc:1240] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 7. Current term is 8. Ops: []
I20260807 01:48:06.747603 27333 raft_consensus.cc:1240] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 2. Current term is 3. Ops: []
I20260807 01:48:06.722606 31662 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:06.724694 31651 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:06.773705 27104 raft_consensus.cc:2410] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260807 01:48:06.733255 31003 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:06.778069 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 69183.1us
I20260807 01:48:06.731381 31769 raft_consensus.cc:3055] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Stepping down as leader of term 5
W20260807 01:48:06.784783 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 51532.7us
I20260807 01:48:06.768491 31662 raft_consensus.cc:3060] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:06.790424 31642 consensus_queue.cc:1048] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260807 01:48:06.792289 31631 consensus_meta.cc:287] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 0.502s user 0.011s sys 0.007s
I20260807 01:48:06.793190 31642 consensus_queue.cc:1048] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:48:06.798575 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:06.799144 31416 consensus_queue.cc:1048] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:48:06.799958 31416 consensus_queue.cc:1243] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer fcf4e45defb94276bcfa68392711dade log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260807 01:48:06.801179 31756 consensus_queue.cc:1048] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:48:06.801993 31756 consensus_queue.cc:1243] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer fcf4e45defb94276bcfa68392711dade log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260807 01:48:06.799628 27109 raft_consensus.cc:2393] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 4: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:06.808341 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:06.779476 31769 raft_consensus.cc:740] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:06.783360 31743 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:06.826457 27063 leader_election.cc:400] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Vote denied by peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429) with higher term. Message: Invalid argument: T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 3. Current term is 4.
I20260807 01:48:06.833102 27063 leader_election.cc:403] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260807 01:48:06.831900 27170 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
W20260807 01:48:06.841033 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 75248.1us
I20260807 01:48:06.816272 26948 catalog_manager.cc:5697] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: config changed from index 5 to 7, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New cstate: current_term: 6 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:48:06.847908 31743 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:06.854573 27287 leader_election.cc:304] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:06.854941 31769 consensus_queue.cc:260] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:06.855810 31639 raft_consensus.cc:2749] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:48:06.856443 31769 raft_consensus.cc:3060] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:06.857648 27287 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:06.862736 31453 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:06.865660 31453 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:06.828402 31771 raft_consensus.cc:2749] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:06.887514 31642 raft_consensus.cc:2749] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260807 01:48:06.895962 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 70056.7us
W20260807 01:48:06.896409 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 55480.4us
I20260807 01:48:06.897176 31749 consensus_queue.cc:1048] T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:48:06.898351 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
W20260807 01:48:06.878690 31810 log.cc:927] Time spent T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.081s user 0.001s sys 0.000s
W20260807 01:48:06.897188 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 147163us
I20260807 01:48:06.903458 31576 consensus_queue.cc:1059] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260807 01:48:06.903893 27067 leader_election.cc:304] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:06.904651 31631 consensus_queue.cc:1059] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 01:48:06.905027 31651 raft_consensus.cc:2749] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:48:06.905565 31604 consensus_queue.cc:1059] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260807 01:48:06.906733 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 103796us
I20260807 01:48:06.907143 31484 raft_consensus.cc:3055] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 01:48:06.907519 31484 raft_consensus.cc:740] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:06.901880 31306 consensus_queue.cc:1048] T b510849c3131475d8c1e032378dc893b P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260807 01:48:06.904364 31576 consensus_queue.cc:1048] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260807 01:48:06.909469 31484 consensus_queue.cc:260] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:06.911087 31720 consensus_queue.cc:1048] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:48:06.917626 27330 raft_consensus.cc:1240] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 7. Current term is 8. Ops: []
I20260807 01:48:06.919087 28715 raft_consensus.cc:2468] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:48:06.920806 27327 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 8.
W20260807 01:48:06.922569 31805 log.cc:927] Time spent T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.300s user 0.000s sys 0.002s
I20260807 01:48:06.905587 31776 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:06.938923 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:06.907317 31749 raft_consensus.cc:3055] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:48:06.944208 31743 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:06.950416 31651 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:06.951280 31651 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:06.953672 31651 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:06.844285 31716 raft_consensus.cc:2749] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429) with higher term. Message: Invalid argument: T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 3. Current term is 4.
I20260807 01:48:06.962688 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:06.968415 28711 raft_consensus.cc:1240] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 11 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 10. Current term is 11. Ops: []
I20260807 01:48:06.975471 28706 raft_consensus.cc:2468] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:48:06.960250 26948 catalog_manager.cc:5697] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 2 to 4, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4). New cstate: current_term: 4 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260807 01:48:06.960928 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 103506us
I20260807 01:48:06.937780 27063 leader_election.cc:304] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:06.989185 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:06.958093 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:06.972069 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:06.965178 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:06.973326 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:06.994997 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 55809.2us
I20260807 01:48:06.941694 31749 raft_consensus.cc:740] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:06.960435 31776 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 11 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:06.970367 31582 raft_consensus.cc:3055] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:48:06.999483 27104 raft_consensus.cc:2393] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 8: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:06.937748 31756 consensus_queue.cc:1048] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.060s
I20260807 01:48:07.009649 27109 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:07.010596 27101 raft_consensus.cc:3060] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:07.027916 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:07.028936 27103 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
W20260807 01:48:07.031378 31632 consensus_meta.cc:287] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.768s user 0.008s sys 0.008s
I20260807 01:48:07.033102 31749 consensus_queue.cc:260] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:07.034708 31749 raft_consensus.cc:3060] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:07.042637 26948 catalog_manager.cc:5697] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 3 to 4, leader changed from 3478729505b046c2a0051c2d0130873f (127.24.4.2) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 4 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } } }
I20260807 01:48:07.044960 26948 catalog_manager.cc:5697] T 965af9378c6a4617b1d50f229ffc2c5b P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:48:07.046258 31771 raft_consensus.cc:2804] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:48:07.046723 31771 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:07.047046 31771 raft_consensus.cc:3060] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:07.050410 27287 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:07.052537 27287 leader_election.cc:304] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:06.954069 31652 raft_consensus.cc:515] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:07.053584 31717 raft_consensus.cc:2764] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260807 01:48:07.054360 31492 raft_consensus.cc:2804] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:48:07.054823 31492 raft_consensus.cc:493] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:48:07.055022 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 142225us
I20260807 01:48:07.055516 31652 leader_election.cc:290] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:07.056273 31492 raft_consensus.cc:3060] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:07.070453 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:07.072042 27104 raft_consensus.cc:2393] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 4: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:07.070489 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:07.004908 31582 raft_consensus.cc:740] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
W20260807 01:48:07.096307 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 195869us
I20260807 01:48:07.107144 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:07.032029 31632 raft_consensus.cc:515] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:07.098264 31639 raft_consensus.cc:2749] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 01:48:07.112267 26948 catalog_manager.cc:5697] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 3 to 4, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4). New cstate: current_term: 4 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:48:07.122380 27063 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
W20260807 01:48:07.131567 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 119695us
I20260807 01:48:07.130563 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:07.112003 31743 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:07.137362 27100 raft_consensus.cc:2393] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 4: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:07.135936 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:07.148038 27287 leader_election.cc:304] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:07.153236 26948 catalog_manager.cc:5697] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 reported cstate change: term changed from 5 to 6, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4). New cstate: current_term: 6 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:48:07.150640 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:07.129793 31396 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:07.178468 27104 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:07.185225 31203 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:07.189112 31203 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:07.190397 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 80692.4us
W20260807 01:48:07.191816 31768 consensus_meta.cc:287] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.673s user 0.014s sys 0.004s
I20260807 01:48:07.192831 31768 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:07.190440 27067 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:07.193892 27063 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
W20260807 01:48:07.194267 31509 consensus_meta.cc:287] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.918s user 0.015s sys 0.004s
I20260807 01:48:07.204169 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:07.184039 31492 raft_consensus.cc:515] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:07.206341 28709 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:48:07.235707 27101 raft_consensus.cc:2468] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:48:07.238390 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:07.200301 31509 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:07.249172 27109 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 8.
I20260807 01:48:07.196637 31662 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:07.258293 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:07.260607 28707 raft_consensus.cc:1240] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 3. Current term is 4. Ops: []
I20260807 01:48:07.204062 31743 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:07.267861 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 74048.9us
I20260807 01:48:07.267331 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
W20260807 01:48:07.271564 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 90015.1us
I20260807 01:48:07.243297 31652 raft_consensus.cc:2749] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:48:07.270192 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:07.283854 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:07.288939 31824 raft_consensus.cc:2749] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:48:07.289660 31654 raft_consensus.cc:2804] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260807 01:48:07.292340 27063 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:07.289407 27108 raft_consensus.cc:1240] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 6. Current term is 7. Ops: []
I20260807 01:48:07.259400 31776 consensus_queue.cc:1059] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260807 01:48:07.315117 31396 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:07.316197 31768 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:48:07.318019 31830 log.cc:927] Time spent T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.121s user 0.001s sys 0.000s
I20260807 01:48:07.318825 27067 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
W20260807 01:48:07.322642 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:07.292788 31654 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:07.293579 31632 leader_election.cc:290] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:48:07.331821 31829 log.cc:927] Time spent T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.086s user 0.000s sys 0.001s
W20260807 01:48:07.335872 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 43628.2us
I20260807 01:48:07.255759 31582 consensus_queue.cc:260] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:07.354038 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 201313us
W20260807 01:48:07.354040 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 211060us
W20260807 01:48:07.356047 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 111954us
W20260807 01:48:07.365547 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 186102us
I20260807 01:48:07.365733 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:07.366447 31662 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:48:07.367141 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 245594us
W20260807 01:48:07.367518 28660 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:07.368139 28660 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:07.369419 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:07.370668 27289 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:07.369976 28660 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:07.372519 31396 raft_consensus.cc:2749] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:07.374033 28660 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:07.374943 31396 raft_consensus.cc:2749] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260807 01:48:07.375205 27214 raft_consensus.cc:1275] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 01:48:07.328069 31654 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 11 FOLLOWER]: Advancing to term 12
I20260807 01:48:07.333988 31716 raft_consensus.cc:2749] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 01:48:07.386426 28655 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
W20260807 01:48:07.387435 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 78793.7us
W20260807 01:48:07.390105 31837 log.cc:927] Time spent T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.063s user 0.000s sys 0.000s
I20260807 01:48:07.390802 31842 raft_consensus.cc:2804] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:07.391327 31842 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:07.391680 31842 raft_consensus.cc:3060] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:07.395161 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:07.395411 27067 leader_election.cc:304] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:07.397017 31716 raft_consensus.cc:2749] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260807 01:48:07.387251 31752 consensus_meta.cc:287] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 0.774s user 0.010s sys 0.005s
I20260807 01:48:07.398612 27063 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:07.398882 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:07.398473 27067 leader_election.cc:304] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
W20260807 01:48:07.390027 31831 log.cc:927] Time spent T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.078s user 0.000s sys 0.001s
I20260807 01:48:07.403679 31717 raft_consensus.cc:2804] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:07.404148 31717 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:07.404506 31717 raft_consensus.cc:3060] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:07.407372 31743 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:07.409361 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:07.411101 27287 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
W20260807 01:48:07.410009 28702 consensus_meta.cc:287] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 0.659s user 0.002s sys 0.014s
W20260807 01:48:07.420339 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 43758.9us
I20260807 01:48:07.389920 31492 leader_election.cc:290] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:07.424757 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:07.426420 28702 raft_consensus.cc:2468] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:48:07.432571 31117 raft_consensus.cc:2804] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 01:48:07.434475 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:07.433425 27230 raft_consensus.cc:1240] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 6. Current term is 7. Ops: []
W20260807 01:48:07.434974 28702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48350 (request call id 858) took 1256 ms (client timeout 1566 ms). Trace:
I20260807 01:48:07.435629 27067 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:07.445779 31764 raft_consensus.cc:2749] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:07.446925 31756 consensus_queue.cc:1059] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:48:07.448632 31756 raft_consensus.cc:3055] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:48:07.449044 31756 raft_consensus.cc:740] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:07.433772 31639 raft_consensus.cc:2749] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260807 01:48:07.449283 31631 raft_consensus.cc:3055] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 01:48:07.415005 31651 raft_consensus.cc:2749] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:48:07.418378 31752 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:07.456110 31752 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:07.441428 31203 raft_consensus.cc:1064] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
W20260807 01:48:07.457099 31203 raft_consensus.cc:1068] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f: Unable to promote non-voter 57bb4985157447f0a3ab680fa0026f12: Illegal state: Leader has not yet committed an operation in its own term
I20260807 01:48:07.457981 28710 raft_consensus.cc:2393] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 5: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:07.460264 27172 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:07.450416 31756 consensus_queue.cc:260] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:07.463771 31756 raft_consensus.cc:3060] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:07.424510 31716 raft_consensus.cc:2804] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:48:07.451000 27212 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Advancing to term 8
W20260807 01:48:07.462743 31827 log.cc:927] Time spent T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.264s user 0.001s sys 0.002s
I20260807 01:48:07.449892 31631 raft_consensus.cc:740] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:07.444837 31582 raft_consensus.cc:3060] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
W20260807 01:48:07.480355 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 56307us
I20260807 01:48:07.410089 31771 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:07.492245 31771 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:07.493170 27102 raft_consensus.cc:1240] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260807 01:48:07.494998 31752 consensus_queue.cc:1048] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:48:07.497082 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:07.497864 27327 raft_consensus.cc:2368] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for earlier term 7. Current term is 8.
I20260807 01:48:07.498029 31492 raft_consensus.cc:493] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:07.498627 31492 raft_consensus.cc:515] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:07.498795 31631 consensus_queue.cc:260] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:07.499679 27336 raft_consensus.cc:1240] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 7. Current term is 8. Ops: []
I20260807 01:48:07.500147 31631 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:07.500795 31492 leader_election.cc:290] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:07.502316 27214 raft_consensus.cc:1240] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 8. Current term is 9. Ops: []
I20260807 01:48:07.479205 31652 raft_consensus.cc:2804] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:48:07.485373 31639 consensus_queue.cc:1059] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260807 01:48:07.468920 31117 raft_consensus.cc:697] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [term 8 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:07.468786 31716 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:48:07.487638 31769 consensus_meta.cc:287] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.631s user 0.016s sys 0.000s
I20260807 01:48:07.482002 31203 raft_consensus.cc:3060] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
W20260807 01:48:07.475167 27063 connection.cc:256] RPC call timeout handler was delayed by -1.2069232151843607s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2808s for a response
I20260807 01:48:07.495343 28660 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:07.508128 27172 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:07.510128 27212 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 8.
I20260807 01:48:07.512295 31117 consensus_queue.cc:237] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:07.513073 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 130975us
I20260807 01:48:07.513244 31716 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:07.510761 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:07.435451 28702 rpcz_store.cc:269] 0807 01:48:06.178586 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:06.178748 (+ 162us) service_pool.cc:224] Handling call
0807 01:48:06.707409 (+528661us) spinlock_profiling.cc:243] Waited 73.8 ms on lock 0x7b54003230a0. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5539c44f 00007f7d55415340 00007f7d55416bbe 00007f7d5542265c 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922
0807 01:48:07.434888 (+727479us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":169658624}
I20260807 01:48:07.522106 27103 raft_consensus.cc:2468] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:07.551610 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:07.526644 31662 raft_consensus.cc:2749] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:07.554005 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:07.554786 27229 raft_consensus.cc:3060] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:07.490036 31654 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:07.563001 31654 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 12 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:07.552868 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:07.567556 27326 raft_consensus.cc:1240] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 6. Current term is 7. Ops: []
I20260807 01:48:07.524248 31764 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:07.569991 27287 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:07.575310 27337 raft_consensus.cc:1275] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260807 01:48:07.576427 27175 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:07.577489 28706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52322 (request call id 700) took 1407 ms. Trace:
I20260807 01:48:07.577888 28706 rpcz_store.cc:276] 0807 01:48:06.169757 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:06.174790 (+ 5033us) service_pool.cc:224] Handling call
0807 01:48:07.573153 (+1398363us) spinlock_profiling.cc:243] Waited 579 ms on lock 0x7b54003a3754. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:07.577427 (+ 4274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1331660672}
I20260807 01:48:07.578908 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:07.579736 27227 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:48:07.573707 28660 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
W20260807 01:48:07.589848 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 110129us
I20260807 01:48:07.550618 31769 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:48:07.594605 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:07.599952 27338 raft_consensus.cc:3060] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:07.603859 31768 raft_consensus.cc:2804] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader election won for term 3
I20260807 01:48:07.604571 31768 raft_consensus.cc:697] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:07.576792 31846 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:07.606915 31846 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:07.609140 31846 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:07.609736 27172 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:07.611992 31662 raft_consensus.cc:2749] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260807 01:48:07.612396 27170 leader_election.cc:336] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.689s (SENT)
I20260807 01:48:07.612917 27170 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:07.613651 27337 raft_consensus.cc:1240] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 3. Current term is 4. Ops: []
I20260807 01:48:07.614490 27229 raft_consensus.cc:2468] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:48:07.615981 31161 raft_consensus.cc:2804] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:48:07.616487 31161 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:07.616326 31769 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:07.616884 31161 raft_consensus.cc:3060] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:07.619092 31639 raft_consensus.cc:2749] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 01:48:07.619145 27172 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:07.620457 31416 raft_consensus.cc:2749] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 01:48:07.619622 31769 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:07.622253 31416 raft_consensus.cc:2804] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:07.622653 31416 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:48:07.623046 31416 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:07.624226 31717 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:07.626297 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:07.627012 27226 raft_consensus.cc:3060] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:07.628592 31662 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
W20260807 01:48:07.623193 27170 leader_election.cc:336] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.722s (SENT)
I20260807 01:48:07.629420 31662 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:07.629814 27170 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:07.631091 31453 raft_consensus.cc:2749] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:48:07.635807 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:07.643738 27338 raft_consensus.cc:2468] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
I20260807 01:48:07.651312 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:07.651165 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 55936.3us
I20260807 01:48:07.652765 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:07.653864 28706 raft_consensus.cc:3060] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
W20260807 01:48:07.656185 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:07.657050 31716 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:07.658639 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:07.660199 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:07.660820 27105 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:48:07.661798 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:07.663362 31816 log.cc:927] Time spent T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.529s user 0.002s sys 0.000s
W20260807 01:48:07.665879 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 114755us
I20260807 01:48:07.605607 31768 consensus_queue.cc:237] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:07.644709 31717 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:07.660372 31509 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:07.674415 27102 raft_consensus.cc:2468] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:48:07.590376 31764 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: false } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:07.671267 31849 raft_consensus.cc:2749] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [term 8 LEADER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260807 01:48:07.695075 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 121444us
I20260807 01:48:07.700657 31662 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:48:07.693152 31861 log.cc:927] Time spent T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.059s user 0.001s sys 0.000s
W20260807 01:48:07.706698 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 131778us
I20260807 01:48:07.701148 31651 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:07.720577 27100 raft_consensus.cc:3055] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 01:48:07.721104 27100 raft_consensus.cc:740] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:07.731029 27226 raft_consensus.cc:2468] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:48:07.734831 28706 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:48:07.739509 27289 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:07.742411 27102 raft_consensus.cc:1240] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 5. Current term is 6. Ops: []
I20260807 01:48:07.740543 31003 raft_consensus.cc:2804] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election won for term 6
W20260807 01:48:07.745416 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 65417.9us
I20260807 01:48:07.723311 31717 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:07.723114 31651 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:07.756166 27102 raft_consensus.cc:1240] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 7. Current term is 8. Ops: []
I20260807 01:48:07.757597 31717 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:07.760982 31743 raft_consensus.cc:3055] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 10 LEADER]: Stepping down as leader of term 10
I20260807 01:48:07.761498 31743 raft_consensus.cc:740] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 10 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:07.769949 31743 consensus_queue.cc:260] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:07.771382 31743 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Advancing to term 11
I20260807 01:48:07.772267 31509 consensus_queue.cc:1059] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260807 01:48:07.772823 27067 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:07.773825 27108 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 630) took 1480 ms. Trace:
I20260807 01:48:07.774241 27108 rpcz_store.cc:276] 0807 01:48:06.293100 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:06.293326 (+ 226us) service_pool.cc:224] Handling call
0807 01:48:06.293668 (+ 342us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:07.320374 (+1026706us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:07.773524 (+453150us) spinlock_profiling.cc:243] Waited 993 ms on lock 0x7b54000b13d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:07.773745 (+ 221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2283507456}
I20260807 01:48:07.775583 31509 raft_consensus.cc:3055] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 01:48:07.775955 31509 raft_consensus.cc:740] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:07.776849 31651 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:07.750532 31003 raft_consensus.cc:697] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:07.776880 31509 consensus_queue.cc:260] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:07.766582 27100 consensus_queue.cc:260] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:07.778134 31003 consensus_queue.cc:237] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:07.782135 31859 log.cc:927] Time spent T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.130s user 0.001s sys 0.000s
I20260807 01:48:07.740126 31161 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:07.783434 31764 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:07.786254 31871 log.cc:927] Time spent T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.055s user 0.000s sys 0.000s
I20260807 01:48:07.774941 31769 raft_consensus.cc:2804] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:48:07.788368 31769 raft_consensus.cc:697] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
W20260807 01:48:07.793213 28660 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:07.793701 28660 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:07.766328 31716 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:07.796315 27287 leader_election.cc:304] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:07.790697 27064 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:07.778554 31509 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:07.781044 31203 raft_consensus.cc:2749] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:48:07.798141 31651 consensus_queue.cc:1059] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260807 01:48:07.806376 31651 raft_consensus.cc:3055] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 8 LEADER]: Stepping down as leader of term 8
I20260807 01:48:07.806770 31651 raft_consensus.cc:740] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:07.807870 27287 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:07.808873 31846 raft_consensus.cc:2804] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:48:07.809382 31846 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:07.809799 31846 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:07.810745 31651 consensus_queue.cc:260] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:07.815168 31651 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:07.811956 31484 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:07.812928 31652 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:07.818495 31652 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Advancing to term 8
W20260807 01:48:07.819454 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:07.820420 31842 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:07.822322 31875 consensus_queue.cc:1059] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260807 01:48:07.823290 31875 consensus_queue.cc:1243] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Peer 57bb4985157447f0a3ab680fa0026f12 log is divergent from this leader: its last log entry 7.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260807 01:48:07.811265 31717 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:07.823940 31764 consensus_queue.cc:1059] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 01:48:07.823377 28702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:34614
I20260807 01:48:07.825187 28702 raft_consensus.cc:606] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 7 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:48:07.798605 31492 raft_consensus.cc:2749] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260807 01:48:07.834389 31860 log.cc:927] Time spent T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.164s user 0.001s sys 0.000s
I20260807 01:48:07.834587 27100 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:07.840298 31416 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:07.843333 31842 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:07.843765 31161 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:07.834676 27064 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
W20260807 01:48:07.845651 27170 leader_election.cc:336] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.700s (SENT)
I20260807 01:48:07.846181 27170 leader_election.cc:304] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:07.845532 27064 leader_election.cc:304] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
W20260807 01:48:07.849431 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 47849.1us
I20260807 01:48:07.802733 31632 raft_consensus.cc:2804] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 01:48:07.854773 27064 leader_election.cc:304] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
W20260807 01:48:07.854315 27170 leader_election.cc:336] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.900s (SENT)
I20260807 01:48:07.858611 27170 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:07.864811 27104 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37478d818c864abc83662e98299718d1"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:48:07.865496 27104 raft_consensus.cc:606] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:48:07.848531 31203 raft_consensus.cc:2749] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:07.862243 31632 raft_consensus.cc:697] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 8 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:07.872475 31632 consensus_queue.cc:237] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:07.859118 31716 raft_consensus.cc:2804] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:48:07.850953 31764 raft_consensus.cc:2804] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:07.884400 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:07.878633 31203 raft_consensus.cc:1064] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:48:07.879595 31654 raft_consensus.cc:2804] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:48:07.885210 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:07.878430 31849 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:07.880002 31716 raft_consensus.cc:697] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:07.879988 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:07.891043 28702 raft_consensus.cc:2368] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6.
I20260807 01:48:07.891224 27229 raft_consensus.cc:1275] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 01:48:07.892259 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:07.850723 31769 consensus_queue.cc:237] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:07.893579 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:07.887892 31764 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:07.895013 31764 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:07.896413 27336 raft_consensus.cc:1240] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 3. Current term is 4. Ops: []
I20260807 01:48:07.898649 28710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
mode: GRACEFUL
new_leader_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:34614
I20260807 01:48:07.899271 28710 raft_consensus.cc:606] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Received request to transfer leadership to TS 3478729505b046c2a0051c2d0130873f
I20260807 01:48:07.893141 27330 raft_consensus.cc:2393] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 4: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
W20260807 01:48:07.874634 28660 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:07.879217 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
W20260807 01:48:07.917686 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 63413.9us
I20260807 01:48:07.894761 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:07.921671 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:48:07.920521 27064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:07.936879 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:07.931159 27213 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 8.
I20260807 01:48:07.941910 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:07.942883 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 60468us
W20260807 01:48:07.889869 31203 raft_consensus.cc:1068] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f: Unable to promote non-voter 57bb4985157447f0a3ab680fa0026f12: Illegal state: Leader has not yet committed an operation in its own term
I20260807 01:48:07.942660 27099 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:48:07.945636 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:07.947187 27100 raft_consensus.cc:1275] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260807 01:48:07.945482 27110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:48:07.948462 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:07.948611 27110 raft_consensus.cc:606] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 7 LEADER]: Received request to transfer leadership to TS 3478729505b046c2a0051c2d0130873f
I20260807 01:48:07.919925 31654 raft_consensus.cc:697] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:07.951400 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:07.952090 27103 raft_consensus.cc:3060] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:07.932861 31416 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:07.956907 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:07.957060 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:07.956810 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:07.957736 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:07.981581 27230 raft_consensus.cc:2368] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for earlier term 4. Current term is 5.
I20260807 01:48:07.958184 31453 raft_consensus.cc:2749] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 01:48:07.960608 31203 consensus_queue.cc:1059] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:48:07.933542 31003 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:07.967737 31716 consensus_queue.cc:237] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:07.969375 31846 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:07.970554 31161 raft_consensus.cc:993] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12: : Instructing follower 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:48:07.970778 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:07.917608 31849 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:07.973327 27110 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:48:07.974790 28655 leader_election.cc:400] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 election: Vote denied by peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429) with higher term. Message: Invalid argument: T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for earlier term 7. Current term is 8.
I20260807 01:48:07.998550 27110 raft_consensus.cc:606] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:48:07.998890 28655 leader_election.cc:403] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260807 01:48:07.974912 28711 raft_consensus.cc:1275] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260807 01:48:07.975454 31484 consensus_queue.cc:1059] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 01:48:07.976701 31654 consensus_queue.cc:237] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:07.978245 28660 leader_election.cc:304] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:07.957883 27109 raft_consensus.cc:1240] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 12 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 11. Current term is 12. Ops: []
I20260807 01:48:08.001936 27213 raft_consensus.cc:1275] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
W20260807 01:48:07.980456 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 46938.8us
I20260807 01:48:07.984266 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:08.003051 27101 raft_consensus.cc:2468] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:07.991686 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:07.918815 31652 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:08.000587 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:08.007946 27105 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:08.013224 31631 raft_consensus.cc:2749] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [term 8 LEADER]: Leader election lost for term 7. Reason: Vote denied by peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429) with higher term. Message: Invalid argument: T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for earlier term 7. Current term is 8.
I20260807 01:48:08.006913 31161 raft_consensus.cc:1081] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Signalling peer 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:48:08.013485 31582 raft_consensus.cc:2749] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:48:08.019362 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:48:08.019263 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 44607us
I20260807 01:48:08.018644 31743 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:08.023155 31416 raft_consensus.cc:3055] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:48:08.023207 31743 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:08.025959 31743 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:48:08.028270 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 44608.6us
I20260807 01:48:08.029414 27170 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:48:08.029634 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260807 01:48:08.030582 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 52471.9us
I20260807 01:48:08.031838 28660 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:08.004660 31003 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:48:08.034814 31003 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:08.035377 31846 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:08.039119 28655 leader_election.cc:304] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:08.041869 27105 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:48:08.031915 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:48:08.044838 27108 raft_consensus.cc:3060] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:07.987597 31764 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:08.053850 27063 leader_election.cc:304] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:08.037032 31416 raft_consensus.cc:740] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
W20260807 01:48:08.065217 31872 log.cc:927] Time spent T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df: Append to log took a long time: real 0.199s user 0.002s sys 0.000s
I20260807 01:48:08.066485 31764 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:08.071493 28655 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:08.068297 28660 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:48:08.055315 31203 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:08.068323 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:08.055308 31484 raft_consensus.cc:2749] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260807 01:48:08.058984 31161 raft_consensus.cc:2749] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 01:48:08.077137 27170 leader_election.cc:304] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:08.079159 27110 raft_consensus.cc:1240] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 7. Current term is 8. Ops: []
I20260807 01:48:08.046423 31776 consensus_queue.cc:1059] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260807 01:48:08.063383 31875 raft_consensus.cc:2749] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:08.076478 31604 raft_consensus.cc:3055] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 01:48:08.072857 31768 raft_consensus.cc:2804] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260807 01:48:08.077437 31884 log.cc:927] Time spent T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.056s user 0.000s sys 0.001s
I20260807 01:48:08.090515 27227 raft_consensus.cc:1240] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6. Ops: []
I20260807 01:48:08.091488 27213 raft_consensus.cc:1275] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 01:48:08.063699 31752 consensus_queue.cc:1048] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.023s
I20260807 01:48:08.083603 31582 raft_consensus.cc:2804] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:48:08.094595 27101 raft_consensus.cc:1240] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 7. Current term is 8. Ops: []
W20260807 01:48:08.094559 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 47758.3us
I20260807 01:48:08.099330 27170 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:48:08.087900 31604 raft_consensus.cc:740] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:08.099552 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:08.100625 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:48:08.107833 27103 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:48:08.106997 27214 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:08.085750 31651 consensus_queue.cc:1048] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260807 01:48:08.095333 31484 raft_consensus.cc:2764] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260807 01:48:08.136010 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 55272.2us
I20260807 01:48:08.097261 31768 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:08.086587 31203 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:08.137874 27172 leader_election.cc:304] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:08.144188 31652 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:48:08.153219 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 90738.6us
I20260807 01:48:08.124110 31582 raft_consensus.cc:697] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
W20260807 01:48:08.156240 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:08.135790 31896 raft_consensus.cc:3060] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:08.157385 27230 raft_consensus.cc:1275] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260807 01:48:08.157981 31901 raft_consensus.cc:2804] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:08.158555 31117 raft_consensus.cc:3055] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 6 LEADER]: Stepping down as leader of term 6
W20260807 01:48:08.156244 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 57017.6us
I20260807 01:48:08.143043 31768 raft_consensus.cc:3060] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:08.150210 31849 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:48:08.174314 28660 leader_election.cc:304] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:08.178128 31908 raft_consensus.cc:2804] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:08.178835 31908 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:08.178895 31888 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:08.179577 31888 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:08.181299 27170 leader_election.cc:304] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:08.186343 31582 consensus_queue.cc:237] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:08.121739 31416 consensus_queue.cc:260] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:08.201746 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:08.190606 27103 raft_consensus.cc:1240] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 6. Current term is 7. Ops: []
W20260807 01:48:08.193276 27067 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:08.165899 31117 raft_consensus.cc:740] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:08.169679 31901 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:08.210878 27230 raft_consensus.cc:1240] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 3. Current term is 4. Ops: []
I20260807 01:48:08.208139 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:48:08.207353 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 132778us
I20260807 01:48:08.211021 27227 raft_consensus.cc:1240] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6. Ops: []
W20260807 01:48:08.221381 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 83568us
I20260807 01:48:08.221933 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:08.252677 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:08.257134 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:08.266331 31888 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:08.266866 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:08.222658 31901 raft_consensus.cc:3060] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
W20260807 01:48:08.268566 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 94398.7us
I20260807 01:48:08.205112 31908 raft_consensus.cc:3060] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:08.429041 27104 raft_consensus.cc:1240] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260807 01:48:08.157095 31604 consensus_queue.cc:260] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:08.431239 31203 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:08.432371 31896 raft_consensus.cc:2749] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:48:08.209939 31776 raft_consensus.cc:2749] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:08.223595 31771 consensus_queue.cc:1059] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
I20260807 01:48:08.224143 31875 consensus_queue.cc:1048] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.039s
I20260807 01:48:08.224001 31632 consensus_queue.cc:1048] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260807 01:48:08.434619 27172 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
W20260807 01:48:08.433888 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 176565us
I20260807 01:48:08.467362 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:08.440056 27214 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
W20260807 01:48:08.440652 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 219404us
W20260807 01:48:08.445606 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:08.448443 27108 raft_consensus.cc:2468] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
W20260807 01:48:08.452986 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 271759us
I20260807 01:48:08.458953 27213 raft_consensus.cc:1275] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 01:48:08.447178 31003 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:08.455453 31756 consensus_queue.cc:1059] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260807 01:48:08.438045 31662 raft_consensus.cc:2749] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:48:08.466176 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:08.461557 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:08.435344 31846 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:08.471189 31003 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:08.481812 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 47276.1us
I20260807 01:48:08.515944 27227 raft_consensus.cc:2393] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 4: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:08.514001 31117 consensus_queue.cc:260] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:08.515815 28660 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:08.477232 28711 raft_consensus.cc:2468] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:08.433876 31769 consensus_queue.cc:1059] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
W20260807 01:48:08.539283 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 66712.3us
I20260807 01:48:08.472870 31752 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:08.481350 31604 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:08.535422 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:08.556396 31117 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Advancing to term 7
W20260807 01:48:08.557796 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 81169.3us
W20260807 01:48:08.564088 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:08.578317 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 107441us
I20260807 01:48:08.577407 31509 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
W20260807 01:48:08.597549 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 70421us
I20260807 01:48:08.574023 31576 raft_consensus.cc:2804] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:48:08.592561 31509 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:08.602111 31576 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:08.602645 31576 raft_consensus.cc:3060] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:08.604298 31509 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:08.617234 31904 log.cc:927] Time spent T 31708844dc4f41feb6a7a2820cfc0e0b P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.071s user 0.003s sys 0.000s
I20260807 01:48:08.623572 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:08.540758 31846 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:08.586709 31875 raft_consensus.cc:3055] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:48:08.627820 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:08.597088 31888 raft_consensus.cc:2804] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:48:08.624185 31631 consensus_queue.cc:1059] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260807 01:48:08.609088 31923 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:08.613483 31453 consensus_queue.cc:1059] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260807 01:48:08.568200 31752 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:08.641788 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:08.637909 31484 consensus_queue.cc:1048] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
W20260807 01:48:08.675783 31862 log.cc:927] Time spent T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.065s user 0.001s sys 0.000s
W20260807 01:48:08.677724 28662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:08.679240 31003 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:08.680593 31003 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:08.684113 31846 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:08.685336 31752 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:08.633184 31928 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:08.634528 31875 raft_consensus.cc:740] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:08.686081 31752 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:08.681155 31003 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:08.637524 31631 raft_consensus.cc:3055] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 01:48:08.642731 31743 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:08.668605 27214 raft_consensus.cc:2393] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 6: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
W20260807 01:48:08.698175 31906 log.cc:927] Time spent T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.123s user 0.001s sys 0.000s
I20260807 01:48:08.641976 31576 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:08.696265 31631 raft_consensus.cc:740] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:08.722082 31743 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:08.722628 31875 consensus_queue.cc:260] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:08.640210 31768 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:08.686682 31928 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:08.726130 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 158938us
I20260807 01:48:08.738404 31117 raft_consensus.cc:493] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:08.729871 27175 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
W20260807 01:48:08.752125 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 310967us
I20260807 01:48:08.765271 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:08.695740 31752 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:08.770037 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:08.766080 27110 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:48:08.773624 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:08.774458 27100 raft_consensus.cc:2393] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 5: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:08.777428 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:08.778203 27110 raft_consensus.cc:3060] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 6
W20260807 01:48:08.779444 28655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:08.783056 27330 raft_consensus.cc:1275] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260807 01:48:08.768040 31764 raft_consensus.cc:993] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3: : Instructing follower 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:48:08.638342 31923 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:08.792507 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 171977us
I20260807 01:48:08.772871 31203 raft_consensus.cc:2804] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:08.795606 28660 leader_election.cc:304] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
W20260807 01:48:08.807248 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 201101us
W20260807 01:48:08.809275 27067 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:08.807384 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:08.811714 27337 raft_consensus.cc:2393] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 8: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:08.791347 31932 raft_consensus.cc:1081] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 7 LEADER]: Signalling peer 3478729505b046c2a0051c2d0130873f to start an election
I20260807 01:48:08.812599 27064 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:08.772322 31117 raft_consensus.cc:515] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:08.826951 27170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:08.827035 27067 leader_election.cc:400] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225) with higher term. Message: Invalid argument: T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6.
I20260807 01:48:08.827893 27067 leader_election.cc:403] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260807 01:48:08.801263 31203 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:08.835502 27337 pending_rounds.cc:85] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade: Aborting all ops after (but not including) 5
I20260807 01:48:08.835944 27337 pending_rounds.cc:107] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260807 01:48:08.836191 27337 raft_consensus.cc:2889] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 01:48:08.810129 31764 raft_consensus.cc:493] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:08.842604 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:08.843400 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:08.843701 27333 raft_consensus.cc:2393] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 8: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
W20260807 01:48:08.843971 27174 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:08.845636 31743 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:08.807408 31631 consensus_queue.cc:260] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:08.810505 31901 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:08.848399 31914 log.cc:927] Time spent T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.087s user 0.000s sys 0.000s
I20260807 01:48:08.849294 31576 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:08.850054 31941 raft_consensus.cc:2749] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:08.851148 31203 raft_consensus.cc:3060] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:08.842990 27331 raft_consensus.cc:1240] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 6. Current term is 7. Ops: []
I20260807 01:48:08.864197 27325 raft_consensus.cc:2468] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:48:08.895462 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:08.967428 27063 leader_election.cc:336] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.503s (SENT)
I20260807 01:48:08.968073 27063 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:08.828876 31771 raft_consensus.cc:2804] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:48:08.979347 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:08.828536 31888 raft_consensus.cc:697] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 7 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
W20260807 01:48:08.986826 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 159271us
I20260807 01:48:08.990944 28711 raft_consensus.cc:1275] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260807 01:48:08.991380 28711 pending_rounds.cc:85] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12: Aborting all ops after (but not including) 5
I20260807 01:48:08.991611 28711 pending_rounds.cc:107] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260807 01:48:08.991860 28711 raft_consensus.cc:2889] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260807 01:48:08.994680 28662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:08.987748 27330 raft_consensus.cc:2468] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
W20260807 01:48:09.005404 27170 leader_election.cc:336] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.967s (SENT)
I20260807 01:48:09.006076 27170 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:08.994026 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:09.024188 27227 raft_consensus.cc:1275] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260807 01:48:09.041186 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 73821us
W20260807 01:48:09.041579 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 46095.5us
I20260807 01:48:09.038681 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:09.053346 27214 raft_consensus.cc:1240] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260807 01:48:09.067953 27110 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:48:09.070976 27230 raft_consensus.cc:1275] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260807 01:48:09.071121 27063 leader_election.cc:336] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.561s (SENT)
I20260807 01:48:08.877398 31764 raft_consensus.cc:515] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.064889 27331 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 8.
I20260807 01:48:09.077644 27216 raft_consensus.cc:1275] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:48:09.077842 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:09.078713 27327 raft_consensus.cc:2393] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 8: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
W20260807 01:48:09.079617 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 284091us
I20260807 01:48:09.081439 28660 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
W20260807 01:48:09.082782 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 77442.8us
I20260807 01:48:09.053809 31771 raft_consensus.cc:697] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 7 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:09.096797 27214 raft_consensus.cc:1240] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 3. Current term is 4. Ops: []
I20260807 01:48:08.989725 31941 consensus_queue.cc:1048] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.091s
W20260807 01:48:09.097143 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:09.102671 27218 raft_consensus.cc:1240] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6. Ops: []
I20260807 01:48:09.130074 27333 raft_consensus.cc:1275] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 01:48:09.103305 31951 raft_consensus.cc:2749] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:09.103730 31943 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:09.107980 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 65894.3us
I20260807 01:48:09.069917 31654 raft_consensus.cc:2749] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225) with higher term. Message: Invalid argument: T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6.
I20260807 01:48:09.081346 31003 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:48:09.128212 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 316563us
I20260807 01:48:09.070724 31881 raft_consensus.cc:2749] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 01:48:09.102543 31898 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:09.098078 27170 leader_election.cc:336] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.560s (SENT)
I20260807 01:48:09.145495 27067 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:09.072654 31729 consensus_queue.cc:1059] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.355s
I20260807 01:48:09.150635 27170 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:09.152547 27067 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:09.153468 31949 raft_consensus.cc:2749] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:09.153892 31945 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:09.154331 27333 raft_consensus.cc:1240] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 3. Current term is 4. Ops: []
I20260807 01:48:09.155555 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:09.156987 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:09.157218 31716 raft_consensus.cc:2749] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260807 01:48:09.157599 27333 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:48:09.157943 31716 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:09.158490 31716 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:09.161320 31899 log.cc:927] Time spent T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.619s user 0.032s sys 0.008s
I20260807 01:48:09.162433 31771 consensus_queue.cc:237] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:09.163304 31923 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:09.162474 31954 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:09.140120 31749 raft_consensus.cc:2749] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 01:48:09.195816 27109 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 659) took 1174 ms. Trace:
I20260807 01:48:09.196318 27109 rpcz_store.cc:276] 0807 01:48:08.040947 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:08.046310 (+ 5363us) service_pool.cc:224] Handling call
0807 01:48:08.061238 (+ 14928us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:08.061454 (+ 216us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:08.061469 (+ 15us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:08.062313 (+ 844us) log.cc:844] Serialized 26 byte log entry
0807 01:48:08.160118 (+ 97805us) raft_consensus.cc:1645] Marking committed up to 5
0807 01:48:08.160159 (+ 41us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:08.160232 (+ 73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:09.160528 (+1000296us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b440002250c. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d5223e855 00007f7d5223f272 00007f7d54fc93bb 00007f7d54fc4c9f 00007f7d54fb8a5e 00007f7d54fb7da6 00007f7d54fb79c8 00007f7d54fc1411 00007f7d54fc13c9 00007f7d54fc1359
0807 01:48:09.165114 (+ 4586us) raft_consensus.cc:1685] finished
0807 01:48:09.165162 (+ 48us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:09.195713 (+ 30551us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24920192,"thread_start_us":70384,"threads_started":1,"wal-append.queue_time_us":175381}
I20260807 01:48:09.194032 31945 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.203735 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:09.204429 27333 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 11 FOLLOWER]: Advancing to term 12
I20260807 01:48:09.205143 31416 raft_consensus.cc:3060] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:09.205710 31945 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:09.165459 31928 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:09.148582 31943 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.209249 31943 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:09.210203 31961 log.cc:927] Time spent T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.105s user 0.000s sys 0.001s
W20260807 01:48:09.210651 31908 consensus_meta.cc:287] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.748s user 0.013s sys 0.000s
I20260807 01:48:09.211148 31908 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:09.213361 31908 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:09.214417 31117 leader_election.cc:290] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:09.215801 31729 raft_consensus.cc:3055] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:48:09.216255 31729 raft_consensus.cc:740] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:09.217126 31729 consensus_queue.cc:260] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.150322 31898 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.195432 31509 consensus_queue.cc:1048] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260807 01:48:09.198825 31954 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.213254 31764 leader_election.cc:290] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:09.222539 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:09.253806 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 73078.5us
I20260807 01:48:09.221287 31756 raft_consensus.cc:2749] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:48:09.255260 31896 raft_consensus.cc:3055] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 11 LEADER]: Stepping down as leader of term 11
I20260807 01:48:09.255702 31896 raft_consensus.cc:740] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:09.256075 31954 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:09.192622 31888 consensus_queue.cc:237] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:09.252967 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 169191us
W20260807 01:48:09.257303 27170 leader_election.cc:336] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.566s (SENT)
W20260807 01:48:09.261762 27170 leader_election.cc:336] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.603s (SENT)
I20260807 01:48:09.262594 27170 leader_election.cc:304] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:09.264194 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:09.264611 27287 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:09.264953 28702 raft_consensus.cc:2468] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 3.
I20260807 01:48:09.266379 27289 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:09.267372 31003 raft_consensus.cc:2804] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:48:09.224730 31935 consensus_queue.cc:1059] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.088s
I20260807 01:48:09.267820 31003 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:09.268242 31003 raft_consensus.cc:3060] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:09.256637 31896 consensus_queue.cc:260] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:09.269109 31898 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:09.269428 31896 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 11 FOLLOWER]: Advancing to term 12
W20260807 01:48:09.272574 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:09.276060 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:09.276857 27331 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 8.
I20260807 01:48:09.277664 27333 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 12.
I20260807 01:48:09.266785 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:09.279764 28707 raft_consensus.cc:2393] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 4: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:09.280593 31764 raft_consensus.cc:493] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:09.279356 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:09.281236 31764 raft_consensus.cc:515] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.282328 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:09.282912 27333 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:09.283555 31764 leader_election.cc:290] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:09.284943 31901 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:09.248226 31729 raft_consensus.cc:3060] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:09.287474 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
W20260807 01:48:09.248169 31604 consensus_meta.cc:287] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 0.678s user 0.012s sys 0.002s
I20260807 01:48:09.291513 28660 leader_election.cc:400] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for earlier term 4. Current term is 5.
I20260807 01:48:09.302731 28711 raft_consensus.cc:1240] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 4. Current term is 5. Ops: []
I20260807 01:48:09.292073 31928 raft_consensus.cc:2749] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 01:48:09.279209 31756 raft_consensus.cc:2749] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:48:09.293506 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:09.293999 31898 raft_consensus.cc:3055] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 01:48:09.295562 28707 raft_consensus.cc:1275] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260807 01:48:09.298347 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:09.292236 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:09.309180 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:09.314610 27338 raft_consensus.cc:3060] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
W20260807 01:48:09.308696 28662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:09.320973 28662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:09.322453 31846 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:09.323084 28708 raft_consensus.cc:2393] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 8: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:09.323805 27229 raft_consensus.cc:1275] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260807 01:48:09.324591 31959 consensus_queue.cc:1048] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 01:48:09.322647 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:09.326438 27327 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:09.301088 27333 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 8.
W20260807 01:48:09.302634 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 51746.2us
I20260807 01:48:09.292414 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:09.301381 27330 raft_consensus.cc:2393] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 4: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:09.218122 31203 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.330714 27338 raft_consensus.cc:2468] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:48:09.333367 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:09.334326 27333 raft_consensus.cc:2393] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 4: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:09.292851 27218 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:41230
I20260807 01:48:09.335789 27218 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:48:09.336524 27218 raft_consensus.cc:3060] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:09.338903 27338 raft_consensus.cc:1275] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260807 01:48:09.339819 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:09.340641 27330 raft_consensus.cc:2393] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 8: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:09.313970 31898 raft_consensus.cc:740] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:09.323120 31846 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.344643 31203 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:09.345026 31846 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:09.345547 27212 raft_consensus.cc:1240] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260807 01:48:09.333990 31901 consensus_queue.cc:1048] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260807 01:48:09.347281 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:09.347950 28706 raft_consensus.cc:2468] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 8.
I20260807 01:48:09.350587 27289 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:09.351629 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:09.352279 28706 raft_consensus.cc:3060] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:09.352635 27327 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
I20260807 01:48:09.354334 27331 raft_consensus.cc:1240] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6. Ops: []
I20260807 01:48:09.355794 31846 raft_consensus.cc:2804] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 01:48:09.370244 28706 raft_consensus.cc:2468] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:48:09.370363 27287 leader_election.cc:304] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:09.377561 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:09.378958 27222 raft_consensus.cc:2468] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:48:09.380231 31953 consensus_queue.cc:1059] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260807 01:48:09.380939 31928 raft_consensus.cc:2749] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:09.371412 31846 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:09.383085 31846 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:09.384122 31898 consensus_queue.cc:260] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.385628 31898 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:09.391261 31651 consensus_queue.cc:1048] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260807 01:48:09.396692 28655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:09.402467 31003 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.400518 27218 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.403160 27175 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:09.406030 27287 leader_election.cc:304] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
W20260807 01:48:09.407069 28655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:09.412086 28655 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:09.413270 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:09.429253 27331 raft_consensus.cc:2468] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:48:09.429117 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:09.432693 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:09.433342 27287 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:09.431316 31003 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:09.435681 27218 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:09.436937 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:09.437665 27327 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 8.
I20260807 01:48:09.426443 31953 raft_consensus.cc:2804] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:48:09.428424 31941 raft_consensus.cc:2804] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:48:09.447415 31003 raft_consensus.cc:2804] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:09.447955 31003 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:09.448308 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:09.449009 28707 raft_consensus.cc:3055] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:48:09.449321 28707 raft_consensus.cc:740] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:09.450477 28707 consensus_queue.cc:260] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.451618 28707 raft_consensus.cc:3060] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:09.432855 31896 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:09.439450 31943 raft_consensus.cc:2749] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 01:48:09.454487 31896 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:09.457259 31896 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:09.446753 31941 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:09.459621 31941 raft_consensus.cc:3060] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:09.448275 31003 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:09.472328 27213 raft_consensus.cc:1240] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 7 LEADER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 6. Current term is 7. Ops: []
I20260807 01:48:09.473654 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:09.474432 27229 raft_consensus.cc:2468] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
I20260807 01:48:09.476403 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:09.477185 27222 raft_consensus.cc:2468] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:48:09.471271 31203 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:09.488433 31203 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:09.489344 27170 leader_election.cc:336] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.709s (SENT)
I20260807 01:48:09.491588 31203 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:09.491788 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:09.488945 31776 consensus_queue.cc:1048] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260807 01:48:09.491312 28660 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:09.494931 31896 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:09.499392 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 64536.7us
I20260807 01:48:09.464812 28707 raft_consensus.cc:2468] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:48:09.503830 27287 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:09.503401 31749 raft_consensus.cc:2804] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 01:48:09.505039 31749 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:09.497872 31971 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:09.505683 31971 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.506080 31896 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.467171 31960 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:09.465236 31953 raft_consensus.cc:697] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 7 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:09.501328 31003 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:48:09.505448 31749 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:09.505362 27287 leader_election.cc:304] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:09.500456 27170 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:09.523046 31003 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:09.507733 28660 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
W20260807 01:48:09.509562 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:09.508616 28655 leader_election.cc:304] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:09.525714 28655 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:09.526484 27175 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:09.505753 31943 raft_consensus.cc:2749] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:09.510263 31582 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:09.527886 31948 raft_consensus.cc:2804] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:48:09.539882 27287 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:09.530248 31776 raft_consensus.cc:2804] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:09.527952 31888 raft_consensus.cc:2749] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:09.532167 31416 raft_consensus.cc:2804] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:48:09.533933 31729 raft_consensus.cc:2804] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:48:09.515080 31960 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.549014 31960 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:09.537060 31716 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:09.556816 31976 log.cc:927] Time spent T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.131s user 0.001s sys 0.000s
I20260807 01:48:09.557945 31416 raft_consensus.cc:697] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:09.554833 31948 raft_consensus.cc:697] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:09.559077 31416 consensus_queue.cc:237] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.559222 31948 consensus_queue.cc:237] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.543002 31776 raft_consensus.cc:493] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:09.550729 31729 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:09.560532 31776 raft_consensus.cc:3060] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:09.560309 31943 raft_consensus.cc:2804] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader election won for term 8
I20260807 01:48:09.561375 31943 raft_consensus.cc:697] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 8 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:09.559964 31896 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:09.562433 31943 consensus_queue.cc:237] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:09.566130 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 76832.1us
W20260807 01:48:09.574168 27170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:09.574522 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 44348.8us
I20260807 01:48:09.579932 28711 raft_consensus.cc:1240] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260807 01:48:09.560725 31729 raft_consensus.cc:3060] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:09.549719 31953 consensus_queue.cc:237] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.581343 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:09.581373 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:09.582808 28707 raft_consensus.cc:2468] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:48:09.584471 27175 leader_election.cc:304] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:09.586612 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:09.557619 31971 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:09.581791 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:09.594934 27229 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
I20260807 01:48:09.539537 31582 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.582473 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:09.554829 31984 raft_consensus.cc:2804] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 01:48:09.605295 27287 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:09.607931 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:09.602463 27330 raft_consensus.cc:2955] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 5 to 7, 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:48:09.614492 31939 raft_consensus.cc:2804] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:48:09.614909 31716 raft_consensus.cc:493] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:09.604105 31764 raft_consensus.cc:476] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Not starting pre-election -- already a leader
I20260807 01:48:09.617340 31939 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:09.617799 31939 raft_consensus.cc:3060] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:09.615662 27327 raft_consensus.cc:1275] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 01:48:09.623267 31003 raft_consensus.cc:2749] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260807 01:48:09.623495 28660 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:09.628286 27287 leader_election.cc:304] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:09.606510 31632 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:48:09.634511 27067 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:09.619107 31909 raft_consensus.cc:3055] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 7 LEADER]: Stepping down as leader of term 7
I20260807 01:48:09.621695 31729 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.607265 31984 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:09.619331 31846 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:09.636693 27170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:09.636968 31652 raft_consensus.cc:3055] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:48:09.634774 31003 raft_consensus.cc:2749] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:09.648895 27287 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:09.645277 31632 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.650528 31990 consensus_queue.cc:1048] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:48:09.624688 31935 raft_consensus.cc:2804] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 01:48:09.615552 31716 raft_consensus.cc:515] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.644399 31909 raft_consensus.cc:740] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
W20260807 01:48:09.654798 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:09.665776 31003 raft_consensus.cc:2749] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260807 01:48:09.667253 27287 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
W20260807 01:48:09.670688 27170 leader_election.cc:336] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.852s (SENT)
I20260807 01:48:09.675340 28707 raft_consensus.cc:1240] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 4. Current term is 5. Ops: []
I20260807 01:48:09.675666 31632 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:09.627156 31764 consensus_queue.cc:1059] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260807 01:48:09.677341 31764 raft_consensus.cc:3055] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 01:48:09.677757 31764 raft_consensus.cc:740] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:09.678683 31764 consensus_queue.cc:260] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.679996 31764 raft_consensus.cc:3060] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:09.681257 31716 leader_election.cc:290] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:09.657441 31984 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:09.690865 31909 consensus_queue.cc:260] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:09.692080 31909 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Advancing to term 8
W20260807 01:48:09.693157 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:09.694029 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:09.665596 31960 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:09.638720 31928 consensus_queue.cc:1059] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260807 01:48:09.696111 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:09.694829 28710 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
I20260807 01:48:09.695951 31901 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:09.664048 31935 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:09.697871 27067 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:09.661887 31652 raft_consensus.cc:740] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:09.695043 31923 raft_consensus.cc:2749] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:09.699266 31901 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.708452 31652 consensus_queue.cc:260] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:09.709713 31652 raft_consensus.cc:3060] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:09.700691 31935 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:09.696069 31960 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:09.711643 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:09.718205 31003 raft_consensus.cc:3055] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Stepping down as leader of term 6
I20260807 01:48:09.719611 31003 raft_consensus.cc:740] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:09.720431 31003 consensus_queue.cc:260] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:09.721627 31003 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:09.709863 31901 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:09.729183 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:09.700951 31846 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:09.717332 31632 raft_consensus.cc:2804] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:09.718814 31729 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:09.740096 31960 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:09.741752 31941 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.744405 31941 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:09.747009 31632 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:09.747722 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:09.748526 27337 raft_consensus.cc:3060] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:09.709619 31749 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.736598 31849 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:09.748157 31632 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:09.764704 27337 raft_consensus.cc:2468] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
W20260807 01:48:09.765978 27170 leader_election.cc:336] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.598s (SENT)
I20260807 01:48:09.766927 28655 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:09.775525 31908 raft_consensus.cc:3055] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:48:09.775906 31768 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:09.777069 27170 leader_election.cc:304] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:09.778787 28708 raft_consensus.cc:2393] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 9: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:09.778712 31943 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:09.791320 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 119776us
I20260807 01:48:09.796075 31749 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:09.785676 31943 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.777683 31908 raft_consensus.cc:740] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:09.784355 31729 raft_consensus.cc:2804] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:48:09.783524 31582 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:09.735118 31923 consensus_queue.cc:1059] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260807 01:48:09.762029 31984 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.802378 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
W20260807 01:48:09.804018 27067 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:09.804804 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:09.786562 31768 raft_consensus.cc:2749] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:09.772969 31849 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.812498 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:09.814440 27330 raft_consensus.cc:2393] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 9: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:09.815747 27337 raft_consensus.cc:1240] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 12 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 11. Current term is 12. Ops: [9.7-11.8]
I20260807 01:48:09.816144 27325 raft_consensus.cc:1240] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260807 01:48:09.825491 28715 raft_consensus.cc:2393] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 9: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:09.790875 31776 raft_consensus.cc:515] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.827656 27328 raft_consensus.cc:1240] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260807 01:48:09.829159 31849 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:09.829432 28710 raft_consensus.cc:1240] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 4. Current term is 5. Ops: []
I20260807 01:48:09.797377 31939 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.834517 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:09.818531 31923 consensus_queue.cc:1243] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer 3262a9d183914b699227fdba0ca5bed3 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260807 01:48:09.843796 27110 raft_consensus.cc:2468] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:48:09.842919 27338 raft_consensus.cc:1240] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260807 01:48:09.818353 31729 raft_consensus.cc:697] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:09.849001 27333 raft_consensus.cc:1240] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260807 01:48:09.849162 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:09.851049 27100 raft_consensus.cc:2410] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260807 01:48:09.860033 27170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:09.860437 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:09.863669 27226 raft_consensus.cc:1275] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260807 01:48:09.850052 27102 raft_consensus.cc:1240] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 7. Current term is 8. Ops: []
I20260807 01:48:09.868948 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:09.869573 31003 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:09.869822 27110 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8.
W20260807 01:48:09.871592 27064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:09.875178 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:09.876085 27102 raft_consensus.cc:2410] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260807 01:48:09.877568 27216 raft_consensus.cc:1240] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6. Ops: []
I20260807 01:48:09.868947 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:09.879443 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:09.880266 27328 raft_consensus.cc:2368] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 3. Current term is 4.
I20260807 01:48:09.881243 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:09.866062 31908 consensus_queue.cc:260] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:09.882697 31993 log.cc:927] Time spent T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.079s user 0.000s sys 0.001s
I20260807 01:48:09.883404 31984 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:09.882756 27212 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40"
dest_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:57662
I20260807 01:48:09.884282 27212 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:48:09.884629 27212 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:09.884794 31939 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:09.885651 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:09.886138 31943 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:09.873852 31960 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:09.886895 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:09.887722 27338 raft_consensus.cc:2368] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 8. Current term is 9.
I20260807 01:48:09.887565 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:09.888427 28710 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 11.
I20260807 01:48:09.874320 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:09.889087 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:09.889490 31729 consensus_queue.cc:237] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.892390 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:09.889859 27289 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:09.893612 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:09.894369 27325 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:48:09.894611 31943 raft_consensus.cc:2804] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260807 01:48:09.895030 31943 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 01:48:09.895262 27063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:09.895727 31776 leader_election.cc:290] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:09.896598 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:09.897338 27331 raft_consensus.cc:2368] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 4. Current term is 5.
I20260807 01:48:09.898105 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
W20260807 01:48:09.897979 27063 leader_election.cc:336] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.708s (SENT)
I20260807 01:48:09.897851 31716 consensus_queue.cc:1059] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260807 01:48:09.900379 31716 raft_consensus.cc:3055] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:48:09.900781 31716 raft_consensus.cc:740] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:09.903575 31752 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:09.904891 31935 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.866707 31764 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:48:09.906751 31764 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.895766 31943 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 11 FOLLOWER]: Advancing to term 12
I20260807 01:48:09.909086 31764 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:09.910326 31632 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.912606 31632 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:09.899881 27063 leader_election.cc:304] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:09.913789 31948 raft_consensus.cc:3055] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:48:09.914253 31948 raft_consensus.cc:740] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:09.915182 31948 consensus_queue.cc:260] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:09.916540 31948 raft_consensus.cc:3060] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:09.916785 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:09.917698 28707 raft_consensus.cc:2393] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 11: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:09.917755 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
W20260807 01:48:09.918455 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:09.877579 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:09.919252 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:09.889127 27333 raft_consensus.cc:3060] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:09.922565 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:09.923272 27226 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 9.
I20260807 01:48:09.888625 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:09.926064 27338 raft_consensus.cc:2368] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 6. Current term is 7.
I20260807 01:48:09.921655 27212 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.924850 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:09.927603 31632 raft_consensus.cc:2749] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:09.988112 31776 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:09.988729 31776 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.990998 31776 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:09.991338 31632 raft_consensus.cc:493] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:09.992005 31632 raft_consensus.cc:515] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.995042 31632 leader_election.cc:290] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:09.995828 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:09.996045 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:09.996795 27338 raft_consensus.cc:2393] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 9: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
W20260807 01:48:09.997833 31996 log.cc:927] Time spent T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.155s user 0.001s sys 0.000s
I20260807 01:48:09.998631 31453 raft_consensus.cc:1064] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:48:09.991317 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:09.999218 31939 raft_consensus.cc:1064] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f: attempting to promote NON_VOTER 57bb4985157447f0a3ab680fa0026f12 to VOTER
I20260807 01:48:09.961290 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:09.957285 27336 raft_consensus.cc:2393] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 6: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:09.895208 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
W20260807 01:48:09.974404 28655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:10.023244 28655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:09.956955 31003 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:09.988251 27102 raft_consensus.cc:1275] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260807 01:48:10.024390 27102 pending_rounds.cc:85] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3: Aborting all ops after (but not including) 7
I20260807 01:48:10.024824 27102 pending_rounds.cc:107] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 6.8
W20260807 01:48:09.975528 27170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:10.027117 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:10.027953 27102 raft_consensus.cc:2917] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Aborting config change with OpId 6.8: Aborted: Op aborted by new leader
I20260807 01:48:10.028553 27102 raft_consensus.cc:2889] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260807 01:48:09.900974 31875 raft_consensus.cc:3060] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
W20260807 01:48:09.938186 28662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:09.928690 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:10.006547 28711 raft_consensus.cc:3060] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:10.038688 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:10.039285 27105 raft_consensus.cc:2468] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:48:10.040210 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:10.041118 27336 raft_consensus.cc:2368] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 6. Current term is 7.
I20260807 01:48:10.042215 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:10.044351 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:09.928099 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:10.045631 27101 raft_consensus.cc:2393] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 7: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:10.034271 31003 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:10.049746 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:10.050463 27338 raft_consensus.cc:3060] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:10.050904 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:10.052134 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:10.052488 31631 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:10.053272 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:10.053910 27325 raft_consensus.cc:2468] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:48:10.037220 31953 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:10.073078 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:10.081071 27330 raft_consensus.cc:2393] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 4: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:10.008234 31943 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:09.956700 31662 consensus_queue.cc:1059] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.117s
W20260807 01:48:10.078243 27170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:10.082322 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:09.886895 31960 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:10.080932 27333 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:48:10.105134 27102 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:52540 (request call id 17) took 6319 ms. Trace:
I20260807 01:48:10.028781 31716 consensus_queue.cc:260] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:10.083878 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:10.074640 27336 raft_consensus.cc:1240] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 3. Current term is 4. Ops: []
I20260807 01:48:10.113195 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:10.113785 27170 leader_election.cc:336] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.947s (SENT)
I20260807 01:48:10.113986 27101 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 8.
I20260807 01:48:10.114400 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:10.061388 31935 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:09.482139 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:48:10.115089 27336 raft_consensus.cc:2468] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 3.
I20260807 01:48:10.116235 27212 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:10.119859 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:10.122100 27105 raft_consensus.cc:2468] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:48:10.121367 27338 raft_consensus.cc:2468] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
W20260807 01:48:10.126756 27170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:10.002053 31453 consensus_queue.cc:237] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 3, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:10.083140 31910 raft_consensus.cc:476] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Not starting pre-election -- already a leader
I20260807 01:48:10.138334 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:10.140017 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:10.145130 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:10.158221 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:10.158569 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:10.159135 27213 raft_consensus.cc:3060] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 5
I20260807 01:48:10.167799 31652 raft_consensus.cc:493] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:10.171269 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:10.171758 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:10.172391 27212 raft_consensus.cc:3060] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:10.174049 31652 raft_consensus.cc:515] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:10.174700 27101 raft_consensus.cc:1240] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 7. Current term is 8. Ops: []
I20260807 01:48:10.171906 27214 raft_consensus.cc:3060] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Advancing to term 9
I20260807 01:48:10.176211 31652 leader_election.cc:290] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:10.177364 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:10.177997 27222 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Advancing to term 9
I20260807 01:48:10.186523 27214 raft_consensus.cc:2468] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 9.
I20260807 01:48:10.187804 27212 raft_consensus.cc:2468] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
I20260807 01:48:10.189745 28660 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:48:10.191246 31898 raft_consensus.cc:2804] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 01:48:10.192200 31898 raft_consensus.cc:697] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [term 9 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:10.194572 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:10.200608 28708 raft_consensus.cc:2393] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 7: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:10.106073 31953 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:10.197757 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
W20260807 01:48:10.202342 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 288141us
I20260807 01:48:10.202752 27336 raft_consensus.cc:3060] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 5
I20260807 01:48:10.203054 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:10.203245 28660 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:10.203728 27338 raft_consensus.cc:3060] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:10.148170 31716 raft_consensus.cc:3060] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:10.208590 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:10.214277 27325 raft_consensus.cc:2393] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 6: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:10.106201 27102 rpcz_store.cc:276] 0807 01:48:03.752899 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:03.753042 (+ 143us) service_pool.cc:224] Handling call
0807 01:48:04.628167 (+875125us) log.cc:844] Serialized 540 byte log entry
0807 01:48:05.795610 (+1167443us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b18002da3c0. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d5b233f31 00007f7d5b2406d1 00007f7d522e5c56 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5539ef06 00007f7d5539887a 00007f7d55397af4
0807 01:48:05.796599 (+ 989us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b54004752a0. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5539ef06 00007f7d5539887a 00007f7d55397af4 00007f7d5537828a 00007f7d5537f133 00007f7d5537f0b9 00007f7d5537f049 00007f7d5537f011 00007f7d5537e1dd 00000000003aa554
0807 01:48:05.961563 (+164964us) spinlock_profiling.cc:243] Waited 29.4 ms on lock 0x7b54004752a0. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d55397522 00007f7d5537828a 00007f7d5537f133 00007f7d5537f0b9 00007f7d5537f049 00007f7d5537f011 00007f7d5537e1dd 00000000003aa554 00000000003aa399 00007f7d52317db5
0807 01:48:10.057184 (+4095621us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":241,"raft.queue_time_us":6671,"raft.run_cpu_time_us":12102,"raft.run_wall_time_us":1777304,"spinlock_wait_cycles":119739520,"thread_start_us":355041,"threads_started":1,"wal-append.queue_time_us":668608,"wal-append.run_cpu_time_us":5682,"wal-append.run_wall_time_us":1463897}
I20260807 01:48:10.218001 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: true dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:10.218789 28702 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:10.224844 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:10.222261 27213 raft_consensus.cc:2468] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
I20260807 01:48:10.225584 27330 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 8.
I20260807 01:48:10.211946 27331 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:48:10.235594 27063 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:10.242415 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:10.250880 27330 raft_consensus.cc:1275] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260807 01:48:10.256101 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:10.256306 27222 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 9.
I20260807 01:48:10.257504 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:10.258314 28660 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:10.259706 28715 raft_consensus.cc:1240] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 6. Current term is 7. Ops: []
I20260807 01:48:10.246191 28655 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:10.264971 27170 leader_election.cc:304] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:10.267267 28702 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8.
I20260807 01:48:10.269018 27338 raft_consensus.cc:2468] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:48:10.223394 31652 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:10.274559 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:10.278354 28708 raft_consensus.cc:1240] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 8. Current term is 9. Ops: []
I20260807 01:48:10.275946 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:10.276839 27170 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
W20260807 01:48:10.277344 27287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:10.227782 31913 raft_consensus.cc:2804] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:48:10.280637 27287 leader_election.cc:304] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:10.279675 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:10.282241 27102 raft_consensus.cc:2393] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 8: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
W20260807 01:48:10.285068 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 49566.5us
I20260807 01:48:10.286284 27063 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:10.288964 26948 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60866 (request call id 218) took 6804 ms. Trace:
I20260807 01:48:10.281016 31756 raft_consensus.cc:2749] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 01:48:10.296609 28713 raft_consensus.cc:1275] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260807 01:48:10.280264 31484 raft_consensus.cc:2804] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 01:48:10.272305 31764 raft_consensus.cc:2804] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:10.297771 31764 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:10.298224 31764 raft_consensus.cc:3060] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:10.298470 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:10.299006 27101 raft_consensus.cc:1275] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 01:48:10.300249 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:10.302856 31943 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 12 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:10.305660 27103 raft_consensus.cc:1275] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260807 01:48:10.306778 32006 log.cc:927] Time spent T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.106s user 0.000s sys 0.000s
I20260807 01:48:10.308413 31898 consensus_queue.cc:237] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:10.311913 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:10.312923 31960 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:10.313393 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:10.314597 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:10.318403 27101 raft_consensus.cc:2393] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 4: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:10.318740 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:10.319872 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:10.289529 26948 rpcz_store.cc:276] 0807 01:48:03.485036 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:03.489946 (+ 4910us) service_pool.cc:224] Handling call
0807 01:48:10.288847 (+6798901us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 01:48:07.609783 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 01:48:07.610512 (+ 729us) write_op.cc:432] Acquiring schema lock in shared mode
0807 01:48:07.610566 (+ 54us) write_op.cc:435] Acquired schema lock
0807 01:48:07.610583 (+ 17us) tablet.cc:662] Decoding operations
0807 01:48:07.611538 (+ 955us) write_op.cc:620] Acquiring the partition lock for write op
0807 01:48:07.611627 (+ 89us) write_op.cc:641] Partition lock acquired for write op
0807 01:48:07.611657 (+ 30us) tablet.cc:685] Acquiring locks for 8 operations
0807 01:48:07.612418 (+ 761us) tablet.cc:701] Row locks acquired
0807 01:48:07.612443 (+ 25us) write_op.cc:260] PREPARE: finished
0807 01:48:07.612625 (+ 182us) write_op.cc:270] Start()
0807 01:48:07.612787 (+ 162us) write_op.cc:276] Timestamp: P: 1786067287612552 usec, L: 0
0807 01:48:07.612828 (+ 41us) op_driver.cc:348] REPLICATION: starting
0807 01:48:07.654554 (+ 41726us) log.cc:844] Serialized 3262 byte log entry
0807 01:48:08.130875 (+476321us) op_driver.cc:464] REPLICATION: finished
0807 01:48:08.190251 (+ 59376us) write_op.cc:301] APPLY: starting
0807 01:48:08.210102 (+ 19851us) tablet.cc:1367] starting BulkCheckPresence
0807 01:48:08.252154 (+ 42052us) tablet.cc:1370] finished BulkCheckPresence
0807 01:48:08.429781 (+177627us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 01:48:08.478383 (+ 48602us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 01:48:08.556984 (+ 78601us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0807 01:48:08.557053 (+ 69us) write_op.cc:312] APPLY: finished
0807 01:48:08.557444 (+ 391us) log.cc:844] Serialized 65 byte log entry
0807 01:48:08.557671 (+ 227us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:48:08.557810 (+ 139us) write_op.cc:454] Released schema lock
0807 01:48:08.557891 (+ 81us) write_op.cc:341] FINISH: Updating metrics
0807 01:48:08.577409 (+ 19518us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:48:08.583709 (+ 6300us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":31732,"reported_tablets":15,"child_traces":[["op",{"apply.queue_time_us":19528,"apply.run_cpu_time_us":50728,"apply.run_wall_time_us":390837,"num_ops":8,"prepare.queue_time_us":204092,"prepare.run_cpu_time_us":34957,"prepare.run_wall_time_us":112415,"raft.queue_time_us":63808,"raft.run_cpu_time_us":2263,"raft.run_wall_time_us":2262,"replication_time_us":517848,"thread_start_us":201418,"threads_started":4,"wal-append.queue_time_us":124917,"wal-append.run_cpu_time_us":19584,"wal-append.run_wall_time_us":1870262}]]}
W20260807 01:48:10.322952 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:10.323596 31953 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:10.326406 31931 raft_consensus.cc:2804] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 01:48:10.326923 31931 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:10.327281 31931 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:10.270305 31203 consensus_queue.cc:1059] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260807 01:48:10.279803 31941 raft_consensus.cc:2804] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 01:48:10.330350 31632 consensus_queue.cc:1048] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:48:10.339105 27099 raft_consensus.cc:2468] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:10.299154 27108 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 8.
W20260807 01:48:10.333880 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 44889us
I20260807 01:48:10.364264 28711 raft_consensus.cc:2468] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:48:10.364639 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:10.365577 27108 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
W20260807 01:48:10.368921 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:10.349677 27067 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:10.372082 27063 leader_election.cc:304] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:10.340212 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:10.351298 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:10.314097 27103 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 10.
W20260807 01:48:10.354817 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 78044.6us
I20260807 01:48:10.374105 27170 leader_election.cc:304] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:10.376088 27170 leader_election.cc:400] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Vote denied by peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429) with higher term. Message: Invalid argument: T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 6. Current term is 7.
I20260807 01:48:10.376487 27170 leader_election.cc:403] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260807 01:48:10.319432 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:10.378547 27101 raft_consensus.cc:2410] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260807 01:48:10.358906 28709 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
I20260807 01:48:10.360124 27336 raft_consensus.cc:2468] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
W20260807 01:48:10.348714 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:10.383117 27170 leader_election.cc:304] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:10.380088 31203 consensus_queue.cc:1243] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer fcf4e45defb94276bcfa68392711dade log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260807 01:48:10.384620 31203 raft_consensus.cc:3060] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:10.385581 31948 raft_consensus.cc:2749] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260807 01:48:10.385344 27170 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:10.387981 27170 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
W20260807 01:48:10.385352 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 104796us
I20260807 01:48:10.330971 31941 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:10.383914 27067 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:10.394431 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:10.394956 27287 leader_election.cc:304] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:10.396027 27103 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:10.396778 27063 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:10.363287 31960 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:10.375258 31484 raft_consensus.cc:697] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [term 9 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:10.384720 31662 raft_consensus.cc:3060] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:10.337026 31913 raft_consensus.cc:697] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:10.330492 31509 consensus_queue.cc:1059] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.065s
I20260807 01:48:10.431384 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:10.395416 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:10.401523 27287 leader_election.cc:304] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:10.401834 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:10.382485 31632 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:10.405555 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:10.446620 27099 raft_consensus.cc:3060] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
W20260807 01:48:10.447039 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:10.382823 31416 raft_consensus.cc:2749] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260807 01:48:10.385469 31908 raft_consensus.cc:3060] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:10.382936 31953 raft_consensus.cc:2804] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:48:10.388723 31888 raft_consensus.cc:2749] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 01:48:10.432186 27101 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
I20260807 01:48:10.391444 31875 raft_consensus.cc:2749] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:48:10.395648 31941 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:10.442899 27287 leader_election.cc:304] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:10.444911 27103 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 9.
W20260807 01:48:10.450873 28662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:10.464790 27099 raft_consensus.cc:2468] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
I20260807 01:48:10.452086 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:10.466228 27102 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:10.466486 31888 raft_consensus.cc:2749] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:10.456244 31953 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:10.469575 31953 raft_consensus.cc:3060] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:10.337354 31652 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:10.472819 31652 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:10.473425 31652 raft_consensus.cc:2804] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:48:10.475728 27110 raft_consensus.cc:2393] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 5: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:10.394595 31756 consensus_queue.cc:1059] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260807 01:48:10.482635 27102 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 9.
I20260807 01:48:10.439983 31948 raft_consensus.cc:2749] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 01:48:10.484575 27287 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:10.400482 31846 consensus_queue.cc:1048] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260807 01:48:10.486361 31003 raft_consensus.cc:3060] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:10.455101 31943 raft_consensus.cc:2749] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 01:48:10.488371 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:10.490361 27327 raft_consensus.cc:1275] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260807 01:48:10.490731 27327 pending_rounds.cc:85] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade: Aborting all ops after (but not including) 2
I20260807 01:48:10.490928 27327 pending_rounds.cc:107] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260807 01:48:10.491122 27327 raft_consensus.cc:2889] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 01:48:10.493747 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:10.473984 31652 raft_consensus.cc:697] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 7 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:10.494511 27327 raft_consensus.cc:2393] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 7: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:10.496101 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:10.498458 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:10.500128 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:10.501253 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:10.503316 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:10.487414 31743 raft_consensus.cc:2749] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:10.455343 31632 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:10.495080 31875 raft_consensus.cc:2804] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 01:48:10.507310 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:10.518872 27337 raft_consensus.cc:2368] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 11. Current term is 12.
W20260807 01:48:10.519886 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 131979us
I20260807 01:48:10.521656 27170 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:10.522701 31768 raft_consensus.cc:1027] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: attempt to promote peer 57bb4985157447f0a3ab680fa0026f12: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260807 01:48:10.511495 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:10.512311 27099 raft_consensus.cc:1240] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 6. Current term is 7. Ops: []
I20260807 01:48:10.526970 27110 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Advancing to term 12
I20260807 01:48:10.506311 27326 raft_consensus.cc:1275] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 01:48:10.456854 31960 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:10.517273 31003 raft_consensus.cc:2749] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:10.525426 27170 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:10.508991 27102 raft_consensus.cc:1240] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 8. Current term is 9. Ops: []
I20260807 01:48:10.552484 27170 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
W20260807 01:48:10.553061 27174 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:10.567952 27104 raft_consensus.cc:1275] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 01:48:10.563009 27105 raft_consensus.cc:2410] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260807 01:48:10.564742 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 180802us
I20260807 01:48:10.579752 27110 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 12.
W20260807 01:48:10.537300 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 140596us
I20260807 01:48:10.527796 31888 raft_consensus.cc:2749] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260807 01:48:10.602386 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 49958.6us
I20260807 01:48:10.602888 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:10.605075 28710 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:48:10.529394 31875 raft_consensus.cc:697] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 8 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:10.611232 27170 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:10.566164 31764 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:10.523454 31484 consensus_queue.cc:237] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:10.518208 31913 consensus_queue.cc:237] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:10.611691 32040 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:10.611169 31888 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:10.636063 31632 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:10.650312 32041 raft_consensus.cc:3060] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:10.650916 31888 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:10.652586 32027 log.cc:927] Time spent T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.054s user 0.000s sys 0.000s
I20260807 01:48:10.653337 31888 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:10.636616 32042 consensus_queue.cc:1048] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:48:10.642875 31960 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:10.644573 31662 raft_consensus.cc:2749] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:48:10.655795 31453 raft_consensus.cc:2804] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election won for term 7
W20260807 01:48:10.656093 32028 log.cc:927] Time spent T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.051s user 0.000s sys 0.000s
W20260807 01:48:10.657284 32025 log.cc:927] Time spent T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.081s user 0.000s sys 0.000s
I20260807 01:48:10.656993 31941 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:10.659545 31941 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:10.660192 31453 raft_consensus.cc:697] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [term 7 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:10.661566 31453 consensus_queue.cc:237] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:10.665786 31764 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:10.599257 31652 consensus_queue.cc:237] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:10.667361 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 56226.7us
I20260807 01:48:10.669943 27063 leader_election.cc:304] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:10.672271 27170 leader_election.cc:304] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:10.678586 27100 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:48:10.679390 27100 raft_consensus.cc:606] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Received request to transfer leadership to TS 3478729505b046c2a0051c2d0130873f
I20260807 01:48:10.679747 27100 raft_consensus.cc:614] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260807 01:48:10.685319 27170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:10.669332 32034 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:10.693081 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 53798.4us
I20260807 01:48:10.700250 31604 raft_consensus.cc:476] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [term 8 LEADER]: Not starting pre-election -- already a leader
I20260807 01:48:10.704624 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:10.684556 32042 raft_consensus.cc:2804] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:48:10.704967 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:10.706404 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:10.707166 27213 raft_consensus.cc:2393] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 6: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:10.707327 27226 raft_consensus.cc:2393] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 5: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
W20260807 01:48:10.709897 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 256227us
I20260807 01:48:10.710490 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:10.711238 27226 raft_consensus.cc:2393] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 8: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:10.712457 27063 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:10.713761 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:10.674539 31960 consensus_queue.cc:1048] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260807 01:48:10.669685 32040 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:10.718439 31631 consensus_meta.cc:287] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 0.665s user 0.010s sys 0.003s
I20260807 01:48:10.721230 32043 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:10.715090 32042 raft_consensus.cc:697] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:10.739966 32052 consensus_queue.cc:1059] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.019s
I20260807 01:48:10.717461 31764 consensus_queue.cc:1059] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260807 01:48:10.748611 31909 raft_consensus.cc:3055] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 7 LEADER]: Stepping down as leader of term 7
I20260807 01:48:10.701365 31875 consensus_queue.cc:237] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:10.757457 27063 leader_election.cc:304] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:10.758836 27063 leader_election.cc:304] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:10.757546 31909 raft_consensus.cc:740] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:10.760217 31909 consensus_queue.cc:260] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:10.761286 31909 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:10.751354 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:10.764359 27222 raft_consensus.cc:2410] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260807 01:48:10.703974 32034 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:10.755339 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:10.744191 31632 raft_consensus.cc:2749] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 01:48:10.739733 31752 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:10.752660 27172 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.4.1:35499 timed out after 1.515s (SENT) [suppressed 43 similar messages]
I20260807 01:48:10.757102 32052 raft_consensus.cc:3055] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Stepping down as leader of term 6
W20260807 01:48:10.760848 32024 log.cc:927] Time spent T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.207s user 0.000s sys 0.002s
I20260807 01:48:10.767271 27213 raft_consensus.cc:2410] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260807 01:48:10.735199 31953 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:10.758900 32041 raft_consensus.cc:2749] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260807 01:48:10.783728 27063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:10.784269 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:10.771867 32052 raft_consensus.cc:740] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:10.752075 31931 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:10.766628 32053 consensus_queue.cc:1048] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260807 01:48:10.798555 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:10.799546 27213 raft_consensus.cc:2368] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5.
W20260807 01:48:10.708298 24592 auto_leader_rebalancer.cc:467] leader transfer for tablet 05e95c9ad127429dbdd5252c0670f22f (from 3262a9d183914b699227fdba0ca5bed3 to 3478729505b046c2a0051c2d0130873f) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 01:48:10.808533 32037 raft_consensus.cc:2749] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:48:10.803411 31960 consensus_queue.cc:1048] T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260807 01:48:10.821659 32034 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:10.830444 27109 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30561911ad3945159907e917cc1bafa7"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:48:10.831146 27109 raft_consensus.cc:606] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:48:10.831486 27109 raft_consensus.cc:614] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 01:48:10.832738 31846 consensus_queue.cc:1048] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 01:48:10.779071 31752 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:10.836107 28706 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:48:10.837198 28710 raft_consensus.cc:1240] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 7. Current term is 8. Ops: []
I20260807 01:48:10.840682 31953 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:10.842710 32052 consensus_queue.cc:260] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:10.844350 31931 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:10.848879 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:10.849848 27229 raft_consensus.cc:2393] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 9: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:10.850559 31203 raft_consensus.cc:2749] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429) with higher term. Message: Invalid argument: T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 6. Current term is 7.
I20260807 01:48:10.850929 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:10.851460 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:10.854185 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:10.803525 32069 raft_consensus.cc:2749] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:48:10.806893 32042 consensus_queue.cc:237] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:10.857827 31003 consensus_queue.cc:1059] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 01:48:10.859305 31003 consensus_queue.cc:1048] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 01:48:10.861022 31743 consensus_queue.cc:1059] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260807 01:48:10.854358 24592 auto_leader_rebalancer.cc:467] leader transfer for tablet 30561911ad3945159907e917cc1bafa7 (from 3262a9d183914b699227fdba0ca5bed3 to fcf4e45defb94276bcfa68392711dade) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 01:48:10.862053 31743 raft_consensus.cc:3055] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Stepping down as leader of term 6
I20260807 01:48:10.862649 31743 raft_consensus.cc:740] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:10.864300 28707 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54eb75662df043488517f09bff1dfcff"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:34614
I20260807 01:48:10.864961 28707 raft_consensus.cc:606] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:48:10.865316 28707 raft_consensus.cc:614] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 01:48:10.866472 31943 raft_consensus.cc:3055] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 8 LEADER]: Stepping down as leader of term 8
I20260807 01:48:10.867245 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:10.850958 31604 raft_consensus.cc:493] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
W20260807 01:48:10.868569 24592 auto_leader_rebalancer.cc:467] leader transfer for tablet 54eb75662df043488517f09bff1dfcff (from 57bb4985157447f0a3ab680fa0026f12 to fcf4e45defb94276bcfa68392711dade) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 01:48:10.868724 31604 raft_consensus.cc:515] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:10.868183 27226 raft_consensus.cc:2393] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 9: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:10.867969 31743 consensus_queue.cc:260] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:10.874337 28710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
mode: GRACEFUL
new_leader_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:34614
I20260807 01:48:10.883452 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:10.884833 28715 raft_consensus.cc:3060] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 11 FOLLOWER]: Advancing to term 12
I20260807 01:48:10.875136 31604 leader_election.cc:290] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:10.876823 28707 raft_consensus.cc:1275] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260807 01:48:10.886852 31716 raft_consensus.cc:3055] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:48:10.887095 31990 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:10.888013 31990 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:10.867110 31943 raft_consensus.cc:740] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 8 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:10.891484 31943 consensus_queue.cc:260] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:10.893236 31943 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:10.894212 28710 raft_consensus.cc:606] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Received request to transfer leadership to TS 3478729505b046c2a0051c2d0130873f
I20260807 01:48:10.894651 28710 raft_consensus.cc:614] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 01:48:10.896729 31416 raft_consensus.cc:3055] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:48:10.897428 31416 raft_consensus.cc:740] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:10.900564 28715 raft_consensus.cc:2468] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 12.
I20260807 01:48:10.845672 32052 raft_consensus.cc:3060] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
W20260807 01:48:10.912583 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 77373.6us
I20260807 01:48:10.917757 27214 raft_consensus.cc:2468] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 7.
I20260807 01:48:10.923439 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:10.924479 27230 raft_consensus.cc:2393] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 9: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:10.926608 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:10.923520 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:10.914881 27287 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:10.930231 31003 raft_consensus.cc:2804] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 12 FOLLOWER]: Leader election won for term 12
I20260807 01:48:10.931046 31003 raft_consensus.cc:697] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 12 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:10.931972 31003 consensus_queue.cc:237] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:10.937332 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:10.939049 27287 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:10.925040 28711 raft_consensus.cc:1275] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEARNER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260807 01:48:10.900854 31939 raft_consensus.cc:1068] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f: Unable to promote non-voter 57bb4985157447f0a3ab680fa0026f12: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }.
Pending config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:10.922219 27175 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:10.887704 31716 raft_consensus.cc:740] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:10.949730 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:10.883569 31743 raft_consensus.cc:3060] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:10.958421 27287 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
W20260807 01:48:10.963378 28662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:10.969386 27287 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
W20260807 01:48:10.969513 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:10.951972 32041 raft_consensus.cc:2804] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader election won for term 8
I20260807 01:48:10.977402 31948 raft_consensus.cc:2749] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260807 01:48:10.960819 31954 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:10.997450 32043 raft_consensus.cc:2749] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:10.971851 32069 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:10.999311 32041 raft_consensus.cc:697] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 8 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:10.989614 31756 consensus_queue.cc:1048] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260807 01:48:10.961174 31846 raft_consensus.cc:2749] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:10.980581 31453 raft_consensus.cc:493] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:10.980132 31896 consensus_queue.cc:1048] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260807 01:48:10.954480 31416 consensus_queue.cc:260] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:11.039664 32041 consensus_queue.cc:237] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:11.041324 31943 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:11.042624 31953 consensus_queue.cc:1175] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "57bb4985157447f0a3ab680fa0026f12" responder_term: 3 status { last_received { term: 3 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 3 index: 6 } }
I20260807 01:48:11.015693 31948 raft_consensus.cc:3060] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:11.043370 31943 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:11.046900 31908 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:11.047726 31908 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:11.014256 32069 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:11.020187 31453 raft_consensus.cc:515] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:11.010949 31954 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:11.028606 31716 consensus_queue.cc:260] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:11.061684 32069 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:11.019884 31632 raft_consensus.cc:2749] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260807 01:48:11.073455 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 104253us
I20260807 01:48:11.072992 31453 leader_election.cc:290] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:11.086578 32034 raft_consensus.cc:3055] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:48:11.084076 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:11.087091 32034 raft_consensus.cc:740] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:11.087730 27214 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:48:11.077968 31752 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:11.112488 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:11.114434 28660 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:11.085312 31908 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:11.087787 27287 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:11.116767 31846 raft_consensus.cc:2749] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:11.117197 27287 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:11.089617 31771 consensus_queue.cc:1059] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260807 01:48:11.091019 27036 auto_rebalancer.cc:912] Removing replace marker failed after 3 inline attempts; will retry on next rebalancer loop iteration: Replica 3262a9d183914b699227fdba0ca5bed3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
W20260807 01:48:11.077425 32023 log.cc:927] Time spent T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.637s user 0.003s sys 0.000s
W20260807 01:48:11.118719 27036 auto_rebalancer.cc:921] Could not move replica: Remote error: Service unavailable: leadership transfer for 00395257beb442e892a4ec18c4c30d95 already in progress
W20260807 01:48:11.119181 27036 auto_rebalancer.cc:342] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 00395257beb442e892a4ec18c4c30d95 already in progress
I20260807 01:48:11.094677 27326 raft_consensus.cc:1275] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 01:48:11.095690 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:11.123344 27336 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:11.088675 32034 consensus_queue.cc:260] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:11.101402 27327 raft_consensus.cc:1275] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260807 01:48:11.118525 27287 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:11.099175 31416 raft_consensus.cc:3060] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:11.108875 31990 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:11.115643 31752 raft_consensus.cc:2804] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:48:11.118095 31003 raft_consensus.cc:2749] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:11.102783 31954 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:11.132751 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:11.134634 27327 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
I20260807 01:48:11.119031 31771 raft_consensus.cc:3055] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 01:48:11.137214 31003 raft_consensus.cc:2749] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 01:48:11.135968 31771 raft_consensus.cc:740] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:11.133085 31752 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:11.141604 31752 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:11.142004 31771 consensus_queue.cc:260] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:11.144161 31771 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:11.159582 27287 leader_election.cc:304] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:11.183914 27287 leader_election.cc:400] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5.
I20260807 01:48:11.184271 27287 leader_election.cc:403] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260807 01:48:11.188562 28655 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:11.190383 27287 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:11.192312 27287 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:11.193137 31003 raft_consensus.cc:2749] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 01:48:11.197063 31003 raft_consensus.cc:2749] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:48:11.197762 27287 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
W20260807 01:48:11.202358 27174 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:11.185053 31990 raft_consensus.cc:493] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:11.203330 31990 raft_consensus.cc:515] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:11.207466 31729 consensus_queue.cc:1048] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260807 01:48:11.208309 27170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:11.211864 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 491456us
I20260807 01:48:11.212392 31846 raft_consensus.cc:3060] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:11.183202 32034 raft_consensus.cc:3060] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:11.136217 32069 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:11.218915 27287 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:11.219331 31003 raft_consensus.cc:2749] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 01:48:11.220216 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:11.220696 32043 consensus_queue.cc:1048] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:48:11.224879 27336 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 9.
I20260807 01:48:11.215507 31776 raft_consensus.cc:2804] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 01:48:11.240675 27287 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:11.195641 31943 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:11.201370 32083 raft_consensus.cc:2749] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:48:11.245566 27287 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:11.249294 27287 leader_election.cc:304] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:48:11.244434 31743 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:11.249487 27064 leader_election.cc:304] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:11.253212 27064 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:11.256772 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:11.257524 27214 raft_consensus.cc:2468] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:48:11.237557 31776 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:11.259514 31776 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:11.262413 31875 raft_consensus.cc:2749] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:48:11.248953 28655 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
W20260807 01:48:11.267030 32084 log.cc:927] Time spent T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.073s user 0.001s sys 0.000s
I20260807 01:48:11.267784 32083 raft_consensus.cc:2804] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:48:11.269398 31990 leader_election.cc:290] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:11.270236 28660 leader_election.cc:304] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:11.276399 31948 raft_consensus.cc:2749] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:11.279838 27064 leader_election.cc:304] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:11.216928 31484 consensus_queue.cc:1048] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
W20260807 01:48:11.285339 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:11.292553 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:11.292332 27064 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:48:11.240159 32053 consensus_queue.cc:1059] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260807 01:48:11.294520 31875 raft_consensus.cc:2749] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:11.286921 31604 raft_consensus.cc:2804] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 01:48:11.282420 31652 raft_consensus.cc:2749] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:11.298498 31604 raft_consensus.cc:697] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 9 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:11.299919 31604 consensus_queue.cc:237] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:11.290267 31954 raft_consensus.cc:2804] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:48:11.303017 31954 raft_consensus.cc:493] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:11.303671 31954 raft_consensus.cc:3060] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:11.301257 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:11.309186 27337 raft_consensus.cc:2468] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:48:11.311760 27063 leader_election.cc:304] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:11.230682 32052 consensus_queue.cc:1059] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260807 01:48:11.234827 32069 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:11.328734 31948 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:11.329656 31948 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:11.330924 32069 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:11.332799 31509 raft_consensus.cc:3055] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:48:11.333330 31948 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:11.335146 24592 auto_leader_rebalancer.cc:467] leader transfer for tablet b3ee061f38f84a71ad3991835a2e56dc (from 57bb4985157447f0a3ab680fa0026f12 to 3478729505b046c2a0051c2d0130873f) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 01:48:11.336788 31509 raft_consensus.cc:740] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:11.338299 31509 consensus_queue.cc:260] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:11.340137 31509 raft_consensus.cc:3060] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:11.284809 31943 raft_consensus.cc:2749] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:48:11.353295 32053 raft_consensus.cc:2749] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:11.354555 32053 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:11.356838 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:11.347223 27063 leader_election.cc:400] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 11 pre-election: Vote denied by peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429) with higher term. Message: Invalid argument: T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 11. Current term is 12.
I20260807 01:48:11.349877 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:11.363552 27063 leader_election.cc:403] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
W20260807 01:48:11.371110 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 445545us
I20260807 01:48:11.371551 28702 raft_consensus.cc:1275] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260807 01:48:11.362915 28707 raft_consensus.cc:2468] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:48:11.373390 27175 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:11.374109 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:11.375123 28707 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 7.
I20260807 01:48:11.369896 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:11.380122 27226 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:48:11.384945 27227 raft_consensus.cc:1275] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260807 01:48:11.387560 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:11.393304 27229 raft_consensus.cc:1275] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260807 01:48:11.310348 32083 raft_consensus.cc:697] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 7 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:11.344538 31875 raft_consensus.cc:2804] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260807 01:48:11.396363 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:11.243573 32076 consensus_queue.cc:1048] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.112s
I20260807 01:48:11.392287 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:11.355472 32053 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:11.412400 27328 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:59474
I20260807 01:48:11.420678 27328 raft_consensus.cc:606] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Received request to transfer leadership
I20260807 01:48:11.421103 27328 raft_consensus.cc:614] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 01:48:11.412756 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
W20260807 01:48:11.385298 32090 log.cc:927] Time spent T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.052s user 0.002s sys 0.000s
I20260807 01:48:11.424628 27289 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
W20260807 01:48:11.345417 31913 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s user 0.018s sys 0.011s
I20260807 01:48:11.413604 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:11.415644 27230 raft_consensus.cc:2393] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 9: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:11.405861 28660 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:48:11.419956 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:11.420807 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:11.425479 28707 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:34614
I20260807 01:48:11.478504 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:11.480567 27222 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:48:11.483196 31003 raft_consensus.cc:2749] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260807 01:48:11.483217 28707 raft_consensus.cc:606] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:48:11.397024 31953 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:11.410481 31875 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 01:48:11.454600 31913 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.697s user 0.019s sys 0.011s
I20260807 01:48:11.399024 32069 consensus_queue.cc:1048] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260807 01:48:11.398471 31632 consensus_queue.cc:1048] T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.089s
I20260807 01:48:11.424037 31943 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:11.425781 32043 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
W20260807 01:48:11.511271 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 262043us
I20260807 01:48:11.427014 32052 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:11.455868 31898 consensus_queue.cc:1048] T 00395257beb442e892a4ec18c4c30d95 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.104s
I20260807 01:48:11.515463 31652 raft_consensus.cc:493] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:48:11.528898 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 104343us
I20260807 01:48:11.455502 31117 consensus_queue.cc:1048] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.058s
I20260807 01:48:11.491561 31875 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Advancing to term 10
I20260807 01:48:11.598263 32048 raft_consensus.cc:2804] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:11.598815 32048 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:11.599170 32048 raft_consensus.cc:3060] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:11.599961 32071 raft_consensus.cc:2749] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429) with higher term. Message: Invalid argument: T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 11. Current term is 12.
I20260807 01:48:11.605841 31849 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:11.606376 32087 log.cc:927] Time spent T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.089s user 0.000s sys 0.001s
I20260807 01:48:11.606673 31849 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:11.610268 27104 raft_consensus.cc:1275] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260807 01:48:11.611033 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:11.611655 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:11.611946 27110 raft_consensus.cc:2410] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260807 01:48:11.611397 31849 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:11.614964 27100 raft_consensus.cc:1275] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260807 01:48:11.611819 31771 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:11.616025 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:11.617467 27099 raft_consensus.cc:1275] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260807 01:48:11.618530 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:11.619182 27103 raft_consensus.cc:2468] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:11.616736 31771 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:11.621081 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:11.622869 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:11.559841 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:11.631415 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:11.632081 27330 raft_consensus.cc:2468] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:48:11.632939 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:11.633476 27330 raft_consensus.cc:3060] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:11.560107 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:11.562268 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:11.634394 27325 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:11.461683 31743 raft_consensus.cc:2749] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 01:48:11.573697 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:11.635201 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:11.636001 27104 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Advancing to term 9
I20260807 01:48:11.494181 31953 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:11.576092 27331 raft_consensus.cc:1240] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260807 01:48:11.576465 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:11.580533 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:11.575222 32052 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:11.582645 32094 log.cc:927] Time spent T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.114s user 0.001s sys 0.000s
I20260807 01:48:11.586321 31652 raft_consensus.cc:515] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:11.570106 31846 raft_consensus.cc:2749] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 4. Current term is 5.
I20260807 01:48:11.654362 31846 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:11.654932 31846 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:11.656919 31846 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:11.592309 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260807 01:48:11.660614 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:48:11.661474 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
W20260807 01:48:11.592649 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:11.595124 32074 raft_consensus.cc:493] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
W20260807 01:48:11.624122 28706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48350 (request call id 896) took 2132 ms (client timeout 1860 ms). Trace:
W20260807 01:48:11.672164 28706 rpcz_store.cc:269] 0807 01:48:09.491396 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:09.491529 (+ 133us) service_pool.cc:224] Handling call
0807 01:48:11.623820 (+2132291us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54001e7b54. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:11.623944 (+ 124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3068466432}
I20260807 01:48:11.543912 31913 raft_consensus.cc:2804] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:11.615451 27100 pending_rounds.cc:85] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3: Aborting all ops after (but not including) 6
I20260807 01:48:11.558645 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:11.683792 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:11.683310 27100 pending_rounds.cc:107] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260807 01:48:11.684614 27100 raft_consensus.cc:2889] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 01:48:11.621950 27110 raft_consensus.cc:2376] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6. Re-sending same reply.
I20260807 01:48:11.685612 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:11.686285 27110 raft_consensus.cc:2468] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:48:11.635094 27333 raft_consensus.cc:2393] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 4: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:11.636574 27213 raft_consensus.cc:2410] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260807 01:48:11.637600 27336 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:48:11.688035 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:11.689271 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:11.689833 27336 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:11.637674 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:11.639557 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:11.691277 27105 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
I20260807 01:48:11.639463 28702 raft_consensus.cc:1275] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 12 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260807 01:48:11.637537 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:11.642923 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:11.643940 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:11.641103 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:11.694465 27331 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
I20260807 01:48:11.695432 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:11.698717 27108 raft_consensus.cc:2376] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 9. Re-sending same reply.
I20260807 01:48:11.646978 31416 consensus_queue.cc:1048] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 01:48:11.650276 27109 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8.
I20260807 01:48:11.538285 31943 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:11.642967 32099 raft_consensus.cc:2804] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:48:11.678817 27172 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:11.704425 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:11.707445 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:11.683995 27328 raft_consensus.cc:2376] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 12. Re-sending same reply.
I20260807 01:48:11.711251 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:11.554803 32043 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:11.683163 31913 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:48:11.680784 27064 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:11.663772 31743 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:11.685423 31752 consensus_meta.cc:287] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 0.543s user 0.012s sys 0.005s
I20260807 01:48:11.718210 27331 raft_consensus.cc:2468] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:48:11.721313 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:11.724933 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:11.739274 27105 raft_consensus.cc:2393] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 9: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:11.739588 27336 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 9.
W20260807 01:48:11.726920 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 48203.6us
W20260807 01:48:11.726245 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 45559.9us
I20260807 01:48:11.720767 27110 raft_consensus.cc:2468] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
I20260807 01:48:11.721833 27100 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 8.
I20260807 01:48:11.742908 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:11.743180 27172 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:11.744241 27336 raft_consensus.cc:2376] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7. Re-sending same reply.
I20260807 01:48:11.744005 27333 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:48:11.745385 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:11.734845 27338 raft_consensus.cc:2468] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
W20260807 01:48:11.721028 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 315237us
I20260807 01:48:11.746429 27330 raft_consensus.cc:2468] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:48:11.738385 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:11.747078 28660 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:11.747612 27328 raft_consensus.cc:2368] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 4. Current term is 5.
I20260807 01:48:11.737061 31901 raft_consensus.cc:2749] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:11.750756 32071 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:11.737473 31752 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:11.751330 32071 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:48:11.751664 31910 consensus_queue.cc:1048] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:48:11.751195 31908 raft_consensus.cc:2804] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:11.754550 31908 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:48:11.755261 31908 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:11.750082 27099 raft_consensus.cc:3060] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:11.755673 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:11.756861 28707 raft_consensus.cc:2368] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 7. Current term is 8.
I20260807 01:48:11.757278 27338 raft_consensus.cc:1240] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6. Ops: []
I20260807 01:48:11.758901 27338 raft_consensus.cc:1275] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260807 01:48:11.754760 27325 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 8.
I20260807 01:48:11.763121 27331 raft_consensus.cc:1240] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 8. Current term is 9. Ops: []
I20260807 01:48:11.719249 31846 consensus_queue.cc:1048] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260807 01:48:11.763242 27325 raft_consensus.cc:1240] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6. Ops: []
I20260807 01:48:11.770393 27325 raft_consensus.cc:1240] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260807 01:48:11.764750 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:11.774690 27105 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:48:11.765836 32040 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:11.737479 31743 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:11.726432 31913 raft_consensus.cc:3060] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:11.723564 32099 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:11.771979 31901 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:11.772483 32069 consensus_queue.cc:1048] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 01:48:11.776800 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:11.764319 31943 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:11.777982 27325 raft_consensus.cc:1240] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6. Ops: []
I20260807 01:48:11.776444 27287 leader_election.cc:304] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:11.779103 27099 raft_consensus.cc:2468] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
I20260807 01:48:11.779088 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:11.802821 27110 raft_consensus.cc:3060] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:11.775174 28660 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:48:11.779796 27331 raft_consensus.cc:1275] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260807 01:48:11.780121 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:11.804334 27105 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 12.
I20260807 01:48:11.780524 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:11.782956 27104 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 9.
I20260807 01:48:11.805371 27101 raft_consensus.cc:2368] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 8. Current term is 9.
I20260807 01:48:11.716336 32105 consensus_queue.cc:1048] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260807 01:48:11.543565 32083 consensus_queue.cc:237] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:11.705586 32074 raft_consensus.cc:515] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:11.800501 27336 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 12.
I20260807 01:48:11.824474 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:48:11.825181 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 50080us
I20260807 01:48:11.796255 32099 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:11.743597 31632 raft_consensus.cc:2804] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:11.808941 27172 leader_election.cc:400] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 8. Current term is 9.
I20260807 01:48:11.784296 31941 raft_consensus.cc:2749] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:11.871680 27064 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:11.871680 27172 leader_election.cc:403] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260807 01:48:11.816008 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:11.817617 27110 raft_consensus.cc:2468] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:48:11.873118 27064 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:11.806128 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:11.825980 28660 leader_election.cc:304] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:48:11.875090 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:11.874071 27104 raft_consensus.cc:2393] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 7: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:11.874096 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:11.877470 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:11.877691 28710 raft_consensus.cc:1275] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260807 01:48:11.874873 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 48978.2us
I20260807 01:48:11.781749 32037 consensus_queue.cc:1048] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260807 01:48:11.819518 31846 raft_consensus.cc:2804] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:11.889719 28660 leader_election.cc:304] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:11.891409 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:11.880175 27064 leader_election.cc:304] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:11.800794 31901 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:11.881182 31898 raft_consensus.cc:2804] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:48:11.881767 31931 raft_consensus.cc:2749] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:11.882270 31416 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:11.802220 31752 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:11.846694 31632 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:11.812460 32053 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:11.886729 27101 raft_consensus.cc:2468] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:11.895295 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:11.886811 31509 raft_consensus.cc:2749] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 01:48:11.902695 27110 raft_consensus.cc:1275] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260807 01:48:11.846921 32106 raft_consensus.cc:2804] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 01:48:11.917794 27110 pending_rounds.cc:85] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3: Aborting all ops after (but not including) 3
I20260807 01:48:11.927407 27110 pending_rounds.cc:107] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260807 01:48:11.927769 27110 raft_consensus.cc:2889] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 01:48:11.910192 28660 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:11.920765 27175 leader_election.cc:400] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Vote denied by peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225) with higher term. Message: Invalid argument: T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 7. Current term is 8.
I20260807 01:48:11.928503 27175 leader_election.cc:403] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260807 01:48:11.885213 27325 raft_consensus.cc:2468] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:48:11.922451 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:11.902240 31960 raft_consensus.cc:2804] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:11.929788 27212 raft_consensus.cc:2410] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260807 01:48:11.929520 31960 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:11.935528 27230 raft_consensus.cc:1275] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260807 01:48:11.940366 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:11.941082 27230 raft_consensus.cc:2468] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:48:11.942655 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:11.943369 27222 raft_consensus.cc:2468] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:48:11.944147 31901 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:11.944490 32052 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:11.925469 31632 raft_consensus.cc:3060] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
W20260807 01:48:11.903122 31776 consensus_meta.cc:287] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 0.643s user 0.011s sys 0.003s
I20260807 01:48:11.887633 31846 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:11.917505 31990 raft_consensus.cc:2804] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:48:11.930362 31960 raft_consensus.cc:3060] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:11.971334 28660 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:11.967530 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:48:11.978538 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 65768.8us
I20260807 01:48:11.959775 31901 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 8
I20260807 01:48:11.915745 31416 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:11.952446 31898 raft_consensus.cc:697] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [term 7 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:11.962209 31846 raft_consensus.cc:3060] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:11.957484 32042 raft_consensus.cc:2749] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 8. Current term is 9.
I20260807 01:48:11.963189 31484 raft_consensus.cc:2749] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260807 01:48:11.994943 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:11.977583 32118 log.cc:927] Time spent T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.139s user 0.002s sys 0.000s
I20260807 01:48:11.966333 31990 raft_consensus.cc:493] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
W20260807 01:48:12.048396 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 127763us
I20260807 01:48:12.054991 31743 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:11.968642 32043 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:11.955137 31453 consensus_queue.cc:1048] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260807 01:48:11.972517 31953 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:11.961499 31776 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:12.062047 27213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57654 (request call id 1924) took 1144 ms. Trace:
W20260807 01:48:12.063408 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:12.102973 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:12.070966 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 91615.9us
I20260807 01:48:12.104712 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:12.104823 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:12.073850 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:12.105697 27216 raft_consensus.cc:2468] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:48:12.075330 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:12.106693 27212 raft_consensus.cc:2393] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 6: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:12.091190 28710 raft_consensus.cc:1275] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 01:48:11.988793 32112 raft_consensus.cc:2804] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:48:11.989058 32106 raft_consensus.cc:697] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 9 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:12.102514 27170 leader_election.cc:304] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:12.101887 27213 rpcz_store.cc:276] 0807 01:48:10.847990 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:10.926270 (+ 78280us) service_pool.cc:224] Handling call
0807 01:48:11.992245 (+1065975us) spinlock_profiling.cc:243] Waited 709 ms on lock 0x7b54002eead4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:11.992401 (+ 156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1630357248}
W20260807 01:48:12.113248 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 304438us
I20260807 01:48:12.057657 32074 leader_election.cc:290] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:12.080037 31990 raft_consensus.cc:3060] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:12.070902 31117 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:12.070966 32107 raft_consensus.cc:493] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:12.056550 32071 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:12.113968 27172 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:12.118867 27170 leader_election.cc:304] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
W20260807 01:48:12.110380 32100 log.cc:927] Time spent T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.323s user 0.000s sys 0.002s
I20260807 01:48:12.127254 27287 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:12.132521 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:12.133335 27213 raft_consensus.cc:2468] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:48:12.139580 27287 leader_election.cc:304] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:48:12.140959 27172 leader_election.cc:304] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:12.143077 27172 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: fcf4e45defb94276bcfa68392711dade
I20260807 01:48:12.110927 32112 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:12.151232 32112 raft_consensus.cc:3060] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:12.151891 32108 raft_consensus.cc:2804] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:48:12.152863 32108 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:12.153442 32108 raft_consensus.cc:3060] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:12.143158 27287 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:12.124037 31203 raft_consensus.cc:2804] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:48:12.130407 32042 raft_consensus.cc:2804] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:48:12.135998 32041 raft_consensus.cc:2804] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260807 01:48:12.144239 31743 raft_consensus.cc:2749] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260807 01:48:12.193732 32116 log.cc:927] Time spent T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.264s user 0.000s sys 0.006s
I20260807 01:48:12.194972 31203 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 01:48:12.218922 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 75754.3us
I20260807 01:48:12.128283 31117 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:12.174964 32040 raft_consensus.cc:2749] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:12.135550 32107 raft_consensus.cc:515] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:12.145336 31910 consensus_queue.cc:1059] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.064s
I20260807 01:48:12.135771 31953 consensus_queue.cc:1048] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260807 01:48:12.145856 31756 consensus_queue.cc:1059] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260807 01:48:12.202256 32105 raft_consensus.cc:2804] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:12.159760 31484 consensus_queue.cc:1048] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260807 01:48:12.205075 32041 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:12.248016 27063 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:12.137285 31898 consensus_queue.cc:237] T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:12.216845 32111 log.cc:927] Time spent T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.315s user 0.001s sys 0.001s
I20260807 01:48:12.208945 32071 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:12.209173 31948 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:12.290306 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 147347us
I20260807 01:48:12.314579 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:12.315354 27330 raft_consensus.cc:2468] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8.
I20260807 01:48:12.315703 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:12.316488 27333 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8.
W20260807 01:48:12.287084 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 176280us
I20260807 01:48:12.223654 31203 raft_consensus.cc:3060] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:12.318838 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:12.319445 31117 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:12.323203 27326 raft_consensus.cc:1275] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260807 01:48:12.325125 32048 consensus_meta.cc:287] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.726s user 0.010s sys 0.001s
I20260807 01:48:12.290638 27327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36604 (request call id 813) took 1063 ms. Trace:
I20260807 01:48:12.250934 32140 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:12.251113 32143 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:12.250698 32135 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:48:12.307199 27108 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.524s user 0.001s sys 0.009s
W20260807 01:48:12.308701 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 60793.3us
I20260807 01:48:12.219517 31453 consensus_queue.cc:1048] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
W20260807 01:48:12.347625 27108 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.565s user 0.001s sys 0.010s
I20260807 01:48:12.334977 32071 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:12.311435 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 99357.2us
I20260807 01:48:12.295601 31910 consensus_queue.cc:1243] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer fcf4e45defb94276bcfa68392711dade log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260807 01:48:12.330109 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:12.330330 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:12.326037 27327 rpcz_store.cc:276] 0807 01:48:11.218534 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:11.218704 (+ 170us) service_pool.cc:224] Handling call
0807 01:48:11.223211 (+ 4507us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:11.223635 (+ 424us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:11.223659 (+ 24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:11.229803 (+ 6144us) log.cc:844] Serialized 26 byte log entry
0807 01:48:11.483003 (+253200us) raft_consensus.cc:1645] Marking committed up to 9
0807 01:48:11.483041 (+ 38us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:11.484378 (+ 1337us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:12.281577 (+797199us) raft_consensus.cc:1685] finished
0807 01:48:12.281602 (+ 25us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:12.281768 (+ 166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3068,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":9509,"thread_start_us":195415,"threads_started":1,"wal-append.queue_time_us":454837}
I20260807 01:48:12.350196 27063 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:12.301928 32105 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:12.351300 32105 raft_consensus.cc:3060] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:12.351596 32071 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:12.351991 27063 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:12.220175 32131 consensus_queue.cc:1059] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.094s
I20260807 01:48:12.354803 27063 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:12.359164 27336 raft_consensus.cc:2393] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 6: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:12.336930 28655 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:12.302834 31416 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:12.308408 32041 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 12 FOLLOWER]: Advancing to term 13
I20260807 01:48:12.367322 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:12.344944 32140 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:12.369429 27336 raft_consensus.cc:1275] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260807 01:48:12.401286 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 46547.3us
I20260807 01:48:12.409600 27330 raft_consensus.cc:2393] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 4: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:12.352264 32071 raft_consensus.cc:2804] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:48:12.417500 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:12.418458 28715 raft_consensus.cc:2393] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 5: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:12.418608 27063 leader_election.cc:400] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Vote denied by peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429) with higher term. Message: Invalid argument: T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 4. Current term is 5.
I20260807 01:48:12.418972 27063 leader_election.cc:403] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260807 01:48:12.419065 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:12.420212 27063 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:12.419893 28711 raft_consensus.cc:2468] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:48:12.422304 31931 raft_consensus.cc:2749] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260807 01:48:12.422521 32132 log.cc:927] Time spent T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.130s user 0.000s sys 0.001s
I20260807 01:48:12.378882 31117 raft_consensus.cc:993] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12: : Instructing follower fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:48:12.423643 27063 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:12.325680 32048 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:12.225124 32106 consensus_queue.cc:237] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:12.365872 32043 raft_consensus.cc:2749] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 01:48:12.428459 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:12.427958 27063 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:12.408600 32053 raft_consensus.cc:2804] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 12 FOLLOWER]: Leader election won for term 12
W20260807 01:48:12.441222 28660 leader_election.cc:336] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Timed out: RequestConsensusVote RPC to 127.24.4.1:35499 timed out after 1.619s (SENT)
I20260807 01:48:12.475867 28660 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
W20260807 01:48:12.473476 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 45167.1us
I20260807 01:48:12.412267 32052 raft_consensus.cc:2804] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 01:48:12.426652 31931 consensus_queue.cc:1048] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.053s
I20260807 01:48:12.480695 27337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36604 (request call id 817) took 1137 ms. Trace:
I20260807 01:48:12.424602 31509 raft_consensus.cc:2749] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429) with higher term. Message: Invalid argument: T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 4. Current term is 5.
I20260807 01:48:12.447755 32069 raft_consensus.cc:2749] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:12.420276 32071 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 01:48:12.482317 31908 consensus_meta.cc:287] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.724s user 0.015s sys 0.000s
I20260807 01:48:12.482129 31509 raft_consensus.cc:2804] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader election won for term 8
W20260807 01:48:12.430634 32125 log.cc:927] Time spent T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.313s user 0.000s sys 0.001s
W20260807 01:48:12.489893 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 48727.6us
I20260807 01:48:12.490649 27222 raft_consensus.cc:2368] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6.
I20260807 01:48:12.344014 31948 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:12.461809 31416 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:12.458886 32146 raft_consensus.cc:3055] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:48:12.481171 27337 rpcz_store.cc:276] 0807 01:48:11.343557 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:11.344063 (+ 506us) service_pool.cc:224] Handling call
0807 01:48:11.381382 (+ 37319us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:11.510579 (+129197us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:11.510614 (+ 35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:11.511144 (+ 530us) log.cc:844] Serialized 26 byte log entry
0807 01:48:11.735011 (+223867us) raft_consensus.cc:1645] Marking committed up to 5
0807 01:48:11.735054 (+ 43us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:11.735128 (+ 74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:12.480459 (+745331us) raft_consensus.cc:1685] finished
0807 01:48:12.480482 (+ 23us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:12.480625 (+ 143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18678,"raft.queue_time_us":105521,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":14266,"thread_start_us":262522,"threads_started":2,"wal-append.queue_time_us":251507}
W20260807 01:48:12.473918 32099 consensus_meta.cc:287] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade: Time spent flushing consensus metadata: real 0.594s user 0.006s sys 0.007s
I20260807 01:48:12.420375 32042 raft_consensus.cc:697] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:12.487419 32071 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:12.379247 31203 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:12.426384 31776 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:12.507794 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:12.462098 31604 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:12.479792 32124 raft_consensus.cc:2804] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:48:12.463917 31935 raft_consensus.cc:1081] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Signalling peer fcf4e45defb94276bcfa68392711dade to start an election
W20260807 01:48:12.546865 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 118990us
I20260807 01:48:12.494773 31484 raft_consensus.cc:2749] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 01:48:12.545363 32146 raft_consensus.cc:740] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:12.520088 32053 raft_consensus.cc:697] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 12 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:12.553748 32151 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:12.564976 27063 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
W20260807 01:48:12.527920 32103 log.cc:927] Time spent T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.369s user 0.001s sys 0.001s
I20260807 01:48:12.501026 31509 raft_consensus.cc:697] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 8 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
W20260807 01:48:12.538611 31954 consensus_meta.cc:287] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 1.234s user 0.011s sys 0.009s
I20260807 01:48:12.512406 32052 raft_consensus.cc:697] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 9 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:12.482936 31908 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:12.552175 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 72473us
I20260807 01:48:12.548687 32144 raft_consensus.cc:3055] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 8 LEADER]: Stepping down as leader of term 8
I20260807 01:48:12.500921 31846 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:12.564055 32124 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:12.564726 31931 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:12.611428 27067 leader_election.cc:304] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:12.366182 32143 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:12.520838 32126 raft_consensus.cc:2804] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:48:12.530745 31416 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:12.546751 32099 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:12.615286 27226 raft_consensus.cc:1240] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 6. Current term is 7. Ops: []
I20260807 01:48:12.617146 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:12.620609 27333 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Advancing to term 9
W20260807 01:48:12.610484 32133 log.cc:927] Time spent T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.314s user 0.000s sys 0.002s
I20260807 01:48:12.410301 32135 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:12.609650 32144 raft_consensus.cc:740] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:12.571426 31604 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:12.611979 32124 raft_consensus.cc:3060] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:12.653967 27333 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 9.
I20260807 01:48:12.656369 27333 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36604
I20260807 01:48:12.615507 32130 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:12.657616 27333 raft_consensus.cc:493] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:48:12.649284 27338 raft_consensus.cc:1240] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 7. Current term is 8. Ops: []
I20260807 01:48:12.661245 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:12.666323 27333 raft_consensus.cc:3060] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:12.582332 32151 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:12.606061 31954 raft_consensus.cc:515] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:12.647909 27326 raft_consensus.cc:2468] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:48:12.675093 28709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48350 (request call id 914) took 1331 ms. Trace:
I20260807 01:48:12.627658 32083 raft_consensus.cc:493] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:12.673178 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 95069.4us
I20260807 01:48:12.676980 28655 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:12.675487 28709 rpcz_store.cc:276] 0807 01:48:11.330758 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:11.336586 (+ 5828us) service_pool.cc:224] Handling call
0807 01:48:11.336937 (+ 351us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:11.337427 (+ 490us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:11.337461 (+ 34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:11.338068 (+ 607us) log.cc:844] Serialized 26 byte log entry
0807 01:48:11.535812 (+197744us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:48:11.535894 (+ 82us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:11.536010 (+ 116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:12.661905 (+1125895us) raft_consensus.cc:1685] finished
0807 01:48:12.661930 (+ 25us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:12.662092 (+ 162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3935,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":136,"thread_start_us":177415,"threads_started":1,"wal-append.queue_time_us":462346}
I20260807 01:48:12.679704 27063 leader_election.cc:304] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:12.640247 32126 raft_consensus.cc:697] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 6 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:12.618702 31484 consensus_queue.cc:1048] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.045s
W20260807 01:48:12.661368 31875 consensus_meta.cc:287] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 1.065s user 0.007s sys 0.009s
I20260807 01:48:12.619037 31931 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:12.696017 27333 raft_consensus.cc:515] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:12.670789 32107 leader_election.cc:290] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:12.704178 27333 leader_election.cc:290] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:12.694662 31652 leader_election.cc:290] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:12.693732 32034 consensus_meta.cc:287] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 1.466s user 0.009s sys 0.010s
W20260807 01:48:12.694314 32137 log.cc:927] Time spent T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.267s user 0.002s sys 0.000s
I20260807 01:48:12.690800 32141 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:12.699061 32163 raft_consensus.cc:2804] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:48:12.700583 31756 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:12.737249 27103 raft_consensus.cc:2468] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:48:12.737248 27109 raft_consensus.cc:1275] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 12 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260807 01:48:12.748032 27109 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1781) took 1027 ms. Trace:
I20260807 01:48:12.744808 32142 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:12.707314 32048 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:12.748654 27109 rpcz_store.cc:276] 0807 01:48:11.720281 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:11.720693 (+ 412us) service_pool.cc:224] Handling call
0807 01:48:12.709519 (+988826us) spinlock_profiling.cc:243] Waited 976 us on lock 0x7b5800024c98. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e084c 00007f7d522e040b 00007f7d5b24c219 00007f7d5a39f640 00007f7d5a467ebb 00007f7d5a467f41 00007f7d5a429dd0 00007f7d5a414739 00007f7d5a4141e2 00007f7d53959d34
0807 01:48:12.731888 (+ 22369us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:12.747663 (+ 15775us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:12.747958 (+ 295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2246272}
W20260807 01:48:12.751107 27170 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s user 0.000s sys 0.016s
I20260807 01:48:12.643111 32042 consensus_queue.cc:237] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:12.688386 32083 raft_consensus.cc:515] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:12.753991 32157 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:12.654946 32146 consensus_queue.cc:260] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:12.671849 31509 consensus_queue.cc:237] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:12.740738 31117 raft_consensus.cc:2804] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 01:48:12.740535 31908 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:48:12.760591 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 111750us
I20260807 01:48:12.730986 32107 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:12.734401 32140 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:12.694751 31875 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:12.735267 32151 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:48:12.751777 27170 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s user 0.000s sys 0.016s
W20260807 01:48:12.756772 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 79847.6us
I20260807 01:48:12.748538 31652 raft_consensus.cc:2804] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:48:12.682279 32053 consensus_queue.cc:237] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:12.749802 32074 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:12.730841 32052 consensus_queue.cc:237] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:12.790772 31203 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:12.681761 32130 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:12.756414 32163 raft_consensus.cc:697] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:12.800767 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
W20260807 01:48:12.806849 31632 consensus_meta.cc:287] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.860s user 0.016s sys 0.000s
W20260807 01:48:12.833679 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:12.795519 31846 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:12.844396 27213 raft_consensus.cc:2393] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 6: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:12.845427 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:12.783541 31604 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:12.800414 31117 raft_consensus.cc:697] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 9 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:12.853018 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:12.782752 32034 raft_consensus.cc:2749] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:12.785882 31875 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:12.795985 31948 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:12.852407 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:12.859414 27100 raft_consensus.cc:3060] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:12.807508 31908 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:12.852428 27172 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s user 0.000s sys 0.011s
I20260807 01:48:12.806919 31931 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:12.757320 31756 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:12.856122 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:12.856526 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:12.872129 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:12.872473 27287 leader_election.cc:304] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:12.873041 27109 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 9.
I20260807 01:48:12.856828 32156 consensus_queue.cc:1048] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260807 01:48:12.857017 27105 raft_consensus.cc:3060] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:12.807407 31652 raft_consensus.cc:493] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:12.861259 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:12.855113 31632 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:12.867345 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 187719us
I20260807 01:48:12.876909 32135 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:12.854686 31416 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:12.796353 32107 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:12.886801 31948 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:48:12.861963 27172 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.515s user 0.000s sys 0.012s
I20260807 01:48:12.863336 31604 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:12.871233 32139 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:12.835453 32074 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:12.770684 32142 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:12.767241 32141 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:12.974696 27100 raft_consensus.cc:2468] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:48:12.929988 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:12.978564 27333 raft_consensus.cc:3060] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:12.931984 31771 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:12.887828 27287 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
W20260807 01:48:12.951723 31901 consensus_meta.cc:287] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.914s user 0.005s sys 0.010s
I20260807 01:48:12.936319 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:12.982115 27174 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:12.982852 28709 raft_consensus.cc:2393] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 8: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:12.906683 31652 raft_consensus.cc:3060] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:12.790563 32157 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:48:12.873099 31908 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:12.938241 31931 consensus_queue.cc:1048] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.006s
I20260807 01:48:12.943813 32107 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:12.907583 31743 raft_consensus.cc:2749] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260807 01:48:12.983266 31913 consensus_meta.cc:287] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 1.184s user 0.014s sys 0.001s
I20260807 01:48:12.955466 27170 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:12.908335 32099 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:12.950515 31203 raft_consensus.cc:493] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:12.960647 32083 leader_election.cc:290] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:12.955922 31948 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:12.967219 31604 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:12.934168 32034 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:12.982556 31901 raft_consensus.cc:2749] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225) with higher term. Message: Invalid argument: T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for earlier term 7. Current term is 8.
I20260807 01:48:12.974447 27105 raft_consensus.cc:1275] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260807 01:48:13.006031 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 50667us
W20260807 01:48:12.982388 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 94654.8us
I20260807 01:48:13.010438 27170 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:12.729143 32144 consensus_queue.cc:260] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:12.928442 31954 leader_election.cc:290] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:12.776381 32126 consensus_queue.cc:237] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:13.011663 31203 raft_consensus.cc:515] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:13.019315 31716 raft_consensus.cc:3060] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:13.022074 32135 raft_consensus.cc:2804] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 01:48:13.011909 31771 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:13.024530 31908 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:12.931828 31117 consensus_queue.cc:237] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:13.026486 31756 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:12.990813 31943 raft_consensus.cc:2804] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260807 01:48:13.035032 32108 consensus_meta.cc:287] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade: Time spent flushing consensus metadata: real 0.881s user 0.013s sys 0.005s
I20260807 01:48:13.009227 31913 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:13.001588 32143 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:12.932780 32163 consensus_queue.cc:237] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:13.015154 32037 raft_consensus.cc:2804] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260807 01:48:13.069193 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:13.100706 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:13.027956 32144 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Advancing to term 9
W20260807 01:48:13.046545 31960 consensus_meta.cc:287] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 1.063s user 0.012s sys 0.004s
I20260807 01:48:13.029644 32034 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:13.087224 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:13.083855 31943 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:13.063107 32135 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:13.110787 28702 raft_consensus.cc:1240] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 6. Current term is 7. Ops: []
I20260807 01:48:12.984398 32139 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:13.113962 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:13.113762 31960 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:13.115388 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:13.116648 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:13.117251 27336 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 9.
I20260807 01:48:13.117913 27110 raft_consensus.cc:2368] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 6. Current term is 7.
I20260807 01:48:13.097939 27213 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 8.
W20260807 01:48:13.104904 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:13.082432 27333 raft_consensus.cc:2468] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
W20260807 01:48:13.093304 32183 log.cc:927] Time spent T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.097s user 0.000s sys 0.001s
I20260807 01:48:13.078715 31771 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:13.107997 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:13.118106 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:13.127876 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:13.120314 27170 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:13.119470 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:13.122757 28655 leader_election.cc:304] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:13.129009 27101 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8.
I20260807 01:48:13.128440 27336 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:13.133720 27170 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:13.092273 32108 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:13.146775 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:13.105170 32037 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:13.120754 31943 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Advancing to term 10
I20260807 01:48:13.120460 32135 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:13.161433 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:13.125963 31931 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:13.182058 27333 raft_consensus.cc:2468] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:48:13.137458 31954 raft_consensus.cc:2804] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader election won for term 5
W20260807 01:48:13.190090 32177 log.cc:927] Time spent T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.105s user 0.000s sys 0.001s
I20260807 01:48:13.143777 31756 raft_consensus.cc:2804] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 01:48:13.166100 32037 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:13.234473 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:13.274663 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:13.254696 27222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57662 (request call id 2811) took 1299 ms. Trace:
I20260807 01:48:13.273897 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:13.247550 27064 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:13.275352 28709 raft_consensus.cc:3060] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 6
I20260807 01:48:13.276079 27100 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Advancing to term 8
W20260807 01:48:13.182025 32105 consensus_meta.cc:287] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade: Time spent flushing consensus metadata: real 0.830s user 0.007s sys 0.014s
I20260807 01:48:13.275400 27222 rpcz_store.cc:276] 0807 01:48:11.954977 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:11.955125 (+ 148us) service_pool.cc:224] Handling call
0807 01:48:13.254395 (+1299270us) spinlock_profiling.cc:243] Waited 516 ms on lock 0x7b54001f77d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:13.254598 (+ 203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1186726656}
I20260807 01:48:13.289492 32141 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:13.290990 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:48:13.207118 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 64770.9us
W20260807 01:48:13.185571 32071 consensus_meta.cc:287] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.621s user 0.015s sys 0.002s
I20260807 01:48:13.294020 32071 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:13.295513 31953 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:13.298383 31953 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:13.302086 31953 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:13.304152 27105 raft_consensus.cc:1240] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260807 01:48:13.307888 32071 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:13.309609 31203 leader_election.cc:290] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:13.273437 31771 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:13.197407 31756 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 01:48:13.313115 31990 consensus_meta.cc:287] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 1.185s user 0.016s sys 0.001s
I20260807 01:48:13.286338 27064 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
W20260807 01:48:13.349931 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 63696.8us
I20260807 01:48:13.350886 27064 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:13.319310 27100 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 8.
I20260807 01:48:13.352521 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:13.353883 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:13.354677 27100 raft_consensus.cc:2468] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8.
I20260807 01:48:13.315742 32105 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:13.228243 32168 log.cc:927] Time spent T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.103s user 0.002s sys 0.000s
W20260807 01:48:13.195667 32175 log.cc:927] Time spent T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.063s user 0.001s sys 0.000s
I20260807 01:48:13.196976 31604 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:13.229488 31908 raft_consensus.cc:2804] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260807 01:48:13.261656 32159 log.cc:927] Time spent T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.514s user 0.000s sys 0.002s
I20260807 01:48:13.332060 28709 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:48:13.430742 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:13.191217 32130 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:13.337268 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:13.333941 31756 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Advancing to term 10
I20260807 01:48:13.271610 31632 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:13.234817 31954 raft_consensus.cc:697] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:13.440729 31908 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:13.487217 31908 raft_consensus.cc:3060] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:13.488065 31954 consensus_queue.cc:237] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:13.495918 27229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57662 (request call id 2802) took 2086 ms. Trace:
I20260807 01:48:13.496470 27229 rpcz_store.cc:276] 0807 01:48:11.409212 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:12.309775 (+900563us) service_pool.cc:224] Handling call
0807 01:48:12.341566 (+ 31791us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:12.341968 (+ 402us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:12.341998 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:12.418048 (+ 76050us) log.cc:844] Serialized 26 byte log entry
0807 01:48:12.766135 (+348087us) raft_consensus.cc:1645] Marking committed up to 2
0807 01:48:12.766741 (+ 606us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:12.766951 (+ 210us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:13.495503 (+728552us) raft_consensus.cc:1685] finished
0807 01:48:13.495530 (+ 27us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:13.495716 (+ 186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":106930,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":13110,"thread_start_us":277508,"threads_started":1,"wal-append.queue_time_us":371221}
I20260807 01:48:13.523713 32180 raft_consensus.cc:2749] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:13.297397 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:13.527486 27101 raft_consensus.cc:2468] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:48:13.347098 32179 consensus_queue.cc:1048] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260807 01:48:13.196671 31931 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:13.544741 31931 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:13.430601 31203 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:13.545809 31203 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:13.438875 32174 raft_consensus.cc:2804] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:13.349309 31771 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:13.425756 32041 consensus_meta.cc:287] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 1.000s user 0.007s sys 0.010s
I20260807 01:48:13.455899 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:13.484108 32169 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:13.370536 31990 raft_consensus.cc:515] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:13.589830 27331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33976 (request call id 2660) took 1274 ms. Trace:
I20260807 01:48:13.452366 31913 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:13.614307 32108 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:13.615299 31943 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:13.619879 27222 raft_consensus.cc:2468] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:48:13.710210 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:13.710879 27222 raft_consensus.cc:3060] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:13.622016 28702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48362 (request call id 844) took 1854 ms. Trace:
I20260807 01:48:13.588757 32130 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:48:13.629034 27099 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52556 (request call id 686) took 1996 ms (client timeout 1619 ms). Trace:
W20260807 01:48:13.639966 28660 leader_election.cc:336] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499): Timed out: RequestConsensusVote RPC to 127.24.4.1:35499 timed out after 1.656s (SENT)
W20260807 01:48:13.714711 27099 rpcz_store.cc:269] 0807 01:48:11.628888 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:11.637282 (+ 8394us) service_pool.cc:224] Handling call
0807 01:48:13.625485 (+1988203us) spinlock_profiling.cc:243] Waited 56.8 ms on lock 0x7b5400357fd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:13.625661 (+ 176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":130525568}
I20260807 01:48:13.699864 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:48:13.717059 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 77132.4us
I20260807 01:48:13.698683 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:13.714378 28702 rpcz_store.cc:276] 0807 01:48:11.767347 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:11.767487 (+ 140us) service_pool.cc:224] Handling call
0807 01:48:11.767854 (+ 367us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:13.111501 (+1343647us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:13.621662 (+510161us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400699454. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:13.621933 (+ 271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3087978752}
I20260807 01:48:13.721064 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:13.728899 28710 raft_consensus.cc:1240] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 3. Current term is 4. Ops: []
I20260807 01:48:13.734203 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:13.641588 32141 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:13.735967 27229 raft_consensus.cc:2410] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260807 01:48:13.698967 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 348172us
I20260807 01:48:13.625382 32174 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:13.728606 32169 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:13.746435 32108 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:13.753834 32108 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:13.734050 32105 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:13.757375 32105 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:13.741706 32174 raft_consensus.cc:3060] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:13.714753 32130 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:13.591457 32157 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:48:13.752207 32124 consensus_meta.cc:287] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 1.073s user 0.007s sys 0.013s
I20260807 01:48:13.747008 32141 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:13.761577 32108 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:13.765194 27222 raft_consensus.cc:2468] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
I20260807 01:48:13.767302 31990 leader_election.cc:290] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:13.769455 31943 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:13.771106 32112 consensus_meta.cc:287] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 1.619s user 0.012s sys 0.009s
W20260807 01:48:13.773105 27327 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.919s user 0.019s sys 0.032s
W20260807 01:48:13.787293 27327 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.938s user 0.019s sys 0.033s
I20260807 01:48:13.776815 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:13.637401 27331 rpcz_store.cc:276] 0807 01:48:12.315118 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:12.317393 (+ 2275us) service_pool.cc:224] Handling call
0807 01:48:12.324501 (+ 7108us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:12.324893 (+ 392us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:12.324922 (+ 29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:12.325409 (+ 487us) log.cc:844] Serialized 26 byte log entry
0807 01:48:12.734044 (+408635us) raft_consensus.cc:1645] Marking committed up to 2
0807 01:48:12.734090 (+ 46us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:12.734198 (+ 108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:13.589534 (+855336us) raft_consensus.cc:1685] finished
0807 01:48:13.589568 (+ 34us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:13.589753 (+ 185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2100,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":11145,"thread_start_us":408163,"threads_started":1,"wal-append.queue_time_us":661718}
I20260807 01:48:13.777376 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:13.592955 31203 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:13.819108 32169 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:13.619212 32039 raft_consensus.cc:2804] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:13.785746 32141 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:13.626371 32034 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:13.762269 32105 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:13.833999 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:13.834919 27331 raft_consensus.cc:3060] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Advancing to term 10
W20260807 01:48:13.791322 31652 consensus_meta.cc:287] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.805s user 0.007s sys 0.011s
I20260807 01:48:13.832451 32130 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:13.780226 32124 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:13.839047 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:13.788429 32112 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:13.839882 32039 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:13.842294 32039 raft_consensus.cc:3060] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:13.839035 28711 raft_consensus.cc:1240] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260807 01:48:13.842882 31453 raft_consensus.cc:3055] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:48:13.843392 31453 raft_consensus.cc:740] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:13.626929 32041 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:13.851366 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:48:13.837858 32144 consensus_meta.cc:287] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.728s user 0.016s sys 0.000s
I20260807 01:48:13.853009 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:13.871922 31716 consensus_meta.cc:287] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.804s user 0.015s sys 0.000s
I20260807 01:48:13.874854 27104 raft_consensus.cc:2368] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for earlier term 5. Current term is 6.
I20260807 01:48:13.875409 27102 raft_consensus.cc:2468] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:48:13.876142 32166 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:13.627784 31771 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:13.857570 27103 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 1787) took 1753 ms. Trace:
I20260807 01:48:13.860621 32074 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:13.861858 32105 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:13.877077 27103 rpcz_store.cc:276] 0807 01:48:12.104088 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:12.104369 (+ 281us) service_pool.cc:224] Handling call
0807 01:48:13.857314 (+1752945us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b54003d77d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:13.857469 (+ 155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1452743424}
I20260807 01:48:13.862682 27226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57654 (request call id 1934) took 1786 ms. Trace:
W20260807 01:48:13.865108 32135 consensus_meta.cc:287] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.673s user 0.003s sys 0.009s
I20260807 01:48:13.866779 28708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48350 (request call id 925) took 1359 ms. Trace:
I20260807 01:48:13.825170 31943 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:13.838762 32124 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:13.886797 28708 rpcz_store.cc:276] 0807 01:48:12.506954 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:12.507119 (+ 165us) service_pool.cc:224] Handling call
0807 01:48:12.507731 (+ 612us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:12.678521 (+170790us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:12.678551 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:12.678594 (+ 43us) raft_consensus.cc:1645] Marking committed up to 7
0807 01:48:12.678625 (+ 31us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:13.866475 (+1187850us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b54007c3dd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:13.866536 (+ 61us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:13.866702 (+ 166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":392301440}
I20260807 01:48:13.861881 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:13.872560 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:13.880167 27226 rpcz_store.cc:276] 0807 01:48:12.076111 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:12.076245 (+ 134us) service_pool.cc:224] Handling call
0807 01:48:12.076569 (+ 324us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:12.620236 (+543667us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:13.862430 (+1242194us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b54001dea54. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:13.862616 (+ 186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1238494592}
I20260807 01:48:13.895449 27229 raft_consensus.cc:2410] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 8 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260807 01:48:13.894634 27230 raft_consensus.cc:2393] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 7: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:13.896932 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:13.907994 27226 raft_consensus.cc:2468] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:48:13.870064 31948 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:13.870983 32112 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:13.916800 27226 raft_consensus.cc:1275] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260807 01:48:13.920344 27226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41230 (request call id 583) took 1118 ms. Trace:
I20260807 01:48:13.844111 31652 raft_consensus.cc:515] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:13.918310 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:13.886060 32179 raft_consensus.cc:493] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:13.920912 27226 rpcz_store.cc:276] 0807 01:48:12.801747 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:13.915794 (+1114047us) service_pool.cc:224] Handling call
0807 01:48:13.916278 (+ 484us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:13.919818 (+ 3540us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:13.920226 (+ 408us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:13.924223 27229 raft_consensus.cc:2393] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 5: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:13.902200 31632 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:13.900523 31716 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:13.931522 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:48:13.942713 27099 raft_consensus.cc:1240] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 4. Current term is 5. Ops: []
I20260807 01:48:13.893282 32135 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:13.935966 27212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41230 (request call id 589) took 1081 ms. Trace:
W20260807 01:48:13.936214 27172 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.992s user 0.024s sys 0.014s
I20260807 01:48:13.926486 27226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41230 (request call id 585) took 1098 ms. Trace:
I20260807 01:48:13.905313 32105 consensus_queue.cc:1059] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
W20260807 01:48:13.953330 27172 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.009s user 0.025s sys 0.014s
I20260807 01:48:13.904392 31943 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:13.954125 27227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57662 (request call id 2810) took 2004 ms. Trace:
I20260807 01:48:13.953140 27212 rpcz_store.cc:276] 0807 01:48:12.854141 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:13.935017 (+1080876us) service_pool.cc:224] Handling call
0807 01:48:13.935372 (+ 355us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:13.935579 (+ 207us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:13.935606 (+ 27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:13.935650 (+ 44us) raft_consensus.cc:1645] Marking committed up to 9
0807 01:48:13.935685 (+ 35us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:13.935751 (+ 66us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:13.935891 (+ 140us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 01:48:13.955953 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 1.01163e+06us
I20260807 01:48:13.891371 27331 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 10.
I20260807 01:48:13.956861 27212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41230 (request call id 590) took 1091 ms. Trace:
I20260807 01:48:13.958856 31960 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:13.955307 27227 rpcz_store.cc:276] 0807 01:48:11.949727 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:11.949918 (+ 191us) service_pool.cc:224] Handling call
0807 01:48:11.964811 (+ 14893us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:12.084820 (+120009us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:12.084860 (+ 40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:12.084919 (+ 59us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:48:12.084959 (+ 40us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:13.953797 (+1868838us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54002ed6d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:13.953853 (+ 56us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:13.954019 (+ 166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":275194624}
I20260807 01:48:13.956578 27226 rpcz_store.cc:276] 0807 01:48:12.827885 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:13.923731 (+1095846us) service_pool.cc:224] Handling call
0807 01:48:13.925726 (+ 1995us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:13.926006 (+ 280us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:13.926034 (+ 28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:13.926086 (+ 52us) raft_consensus.cc:1645] Marking committed up to 8
0807 01:48:13.926128 (+ 42us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:13.926225 (+ 97us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:13.926378 (+ 153us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 01:48:13.936132 32161 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.023s user 0.028s sys 0.082s
I20260807 01:48:13.892287 31453 consensus_queue.cc:260] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:13.975384 31943 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:13.978055 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:13.983143 27331 raft_consensus.cc:2468] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8.
I20260807 01:48:14.008432 27226 raft_consensus.cc:1275] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 01:48:14.009301 27226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41230 (request call id 593) took 1094 ms. Trace:
I20260807 01:48:14.009675 27226 rpcz_store.cc:276] 0807 01:48:12.915459 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:13.979476 (+1064017us) service_pool.cc:224] Handling call
0807 01:48:13.982529 (+ 3053us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:14.009000 (+ 26471us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.009236 (+ 236us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 01:48:13.976750 32161 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.085s user 0.028s sys 0.083s
I20260807 01:48:13.955364 32041 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 13 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:13.939900 31632 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:13.978034 32174 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:13.979681 32144 raft_consensus.cc:493] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:14.029845 31453 raft_consensus.cc:3060] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:13.946753 31716 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:14.050820 32174 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:13.982573 31953 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:14.053046 31953 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:14.056135 31756 consensus_meta.cc:287] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.553s user 0.010s sys 0.005s
I20260807 01:48:14.049849 32144 raft_consensus.cc:515] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:14.008440 32105 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:13.876880 32166 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:13.957275 27212 rpcz_store.cc:276] 0807 01:48:12.865223 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:13.955981 (+1090758us) service_pool.cc:224] Handling call
0807 01:48:13.956337 (+ 356us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:13.956531 (+ 194us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:13.956555 (+ 24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:13.956598 (+ 43us) raft_consensus.cc:1645] Marking committed up to 10
0807 01:48:13.956637 (+ 39us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:13.956694 (+ 57us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:13.956802 (+ 108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:14.065418 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:14.009409 32142 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:14.070267 27108 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.095s user 0.011s sys 0.020s
I20260807 01:48:14.056749 31756 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:13.935453 32179 raft_consensus.cc:515] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:14.076622 31908 consensus_meta.cc:287] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.588s user 0.014s sys 0.000s
I20260807 01:48:14.077275 32130 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:14.082942 27108 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.111s user 0.012s sys 0.020s
I20260807 01:48:14.085568 27331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33976 (request call id 2666) took 1051 ms. Trace:
W20260807 01:48:14.089725 27170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:14.116849 27170 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:14.075618 32139 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:14.082492 27226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41230 (request call id 595) took 1162 ms. Trace:
I20260807 01:48:14.112591 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:14.113512 31203 consensus_queue.cc:1048] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:48:14.130200 27108 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1783) took 2381 ms. Trace:
W20260807 01:48:14.130367 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:14.131389 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:14.117727 27331 rpcz_store.cc:276] 0807 01:48:13.033737 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:13.986540 (+952803us) service_pool.cc:224] Handling call
0807 01:48:14.085474 (+ 98934us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:14.120839 27213 raft_consensus.cc:2393] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 10: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:14.126070 27063 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:14.129288 27330 raft_consensus.cc:2393] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 5: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:14.129523 27226 rpcz_store.cc:276] 0807 01:48:12.920607 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.069703 (+1149096us) service_pool.cc:224] Handling call
0807 01:48:14.081954 (+ 12251us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:14.082142 (+ 188us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:14.082197 (+ 55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:14.082241 (+ 44us) raft_consensus.cc:1645] Marking committed up to 7
0807 01:48:14.082271 (+ 30us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.082324 (+ 53us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:14.082430 (+ 106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:14.133982 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:14.134874 32069 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:14.135818 32069 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:14.137046 28711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48350 (request call id 923) took 1631 ms. Trace:
I20260807 01:48:14.137475 28711 rpcz_store.cc:276] 0807 01:48:12.506134 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:12.518563 (+ 12429us) service_pool.cc:224] Handling call
0807 01:48:12.519044 (+ 481us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:13.843137 (+1324093us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.136736 (+293599us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54002b7d54. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:14.136971 (+ 235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3035068672}
I20260807 01:48:14.138255 31943 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:14.139411 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:14.134841 27337 raft_consensus.cc:2393] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 5: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:14.129635 27170 leader_election.cc:304] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:14.130668 27108 rpcz_store.cc:276] 0807 01:48:11.734103 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:11.734308 (+ 205us) service_pool.cc:224] Handling call
0807 01:48:11.734645 (+ 337us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:11.734853 (+ 208us) raft_consensus.cc:1534] Early marking committed up to index 6
0807 01:48:12.348689 (+613836us) log.cc:844] Serialized 14 byte log entry
0807 01:48:12.971912 (+623223us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b18002a2620. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d5b233f31 00007f7d5b2406d1 00007f7d52291d1a 00007f7d5ba255d9 00007f7d5231a3a0 00007f7d52314e94 00007f7d52317542 00007f7d54fb73fb 00007f7d54fbcf13 00007f7d54fbd42a 00007f7d55428e0a
0807 01:48:14.115171 (+1143259us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:14.115238 (+ 67us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:48:14.115296 (+ 58us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.115379 (+ 83us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:14.115505 (+ 126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61617,"prepare.queue_time_us":673381,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":2996,"raft.queue_time_us":547,"spinlock_wait_cycles":6851456,"thread_start_us":1708288,"threads_started":2}
I20260807 01:48:14.141647 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:14.142518 27330 raft_consensus.cc:2393] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 10: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:14.142750 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:14.142953 27330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33976 (request call id 2668) took 1108 ms. Trace:
I20260807 01:48:14.140069 27333 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:48:14.143790 27333 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59490 (request call id 2685) took 1112 ms. Trace:
I20260807 01:48:14.067016 31652 leader_election.cc:290] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:14.083225 32174 raft_consensus.cc:493] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:14.149698 27226 raft_consensus.cc:1275] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260807 01:48:14.150424 27337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59490 (request call id 2684) took 1120 ms. Trace:
I20260807 01:48:14.150836 27337 rpcz_store.cc:276] 0807 01:48:13.030408 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.116874 (+1086466us) service_pool.cc:224] Handling call
0807 01:48:14.150352 (+ 33478us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:14.143302 27330 rpcz_store.cc:276] 0807 01:48:13.034264 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.141375 (+1107111us) service_pool.cc:224] Handling call
0807 01:48:14.142892 (+ 1517us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:14.147902 27063 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:14.144073 27333 rpcz_store.cc:276] 0807 01:48:13.030953 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.139045 (+1108092us) service_pool.cc:224] Handling call
0807 01:48:14.143732 (+ 4687us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 01:48:14.088419 31898 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.281s user 0.004s sys 0.034s
I20260807 01:48:14.166509 27226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57654 (request call id 1939) took 1129 ms. Trace:
I20260807 01:48:14.166981 27226 rpcz_store.cc:276] 0807 01:48:13.036681 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.141514 (+1104833us) service_pool.cc:224] Handling call
0807 01:48:14.141840 (+ 326us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:14.150229 (+ 8389us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.166433 (+ 16204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:14.060433 32105 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:14.184998 31203 raft_consensus.cc:2804] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:48:14.185714 31203 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:14.186216 31203 raft_consensus.cc:3060] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:14.117017 32180 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
W20260807 01:48:14.172612 31898 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.370s user 0.004s sys 0.035s
I20260807 01:48:14.172819 32074 raft_consensus.cc:2804] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Leader election won for term 10
I20260807 01:48:14.163847 32139 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:14.164740 31948 raft_consensus.cc:493] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:14.139110 31943 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:14.126967 31908 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:14.202513 32179 leader_election.cc:290] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:14.214282 32139 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:14.141084 32130 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:14.232972 31632 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:14.175551 32174 raft_consensus.cc:515] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:14.210287 32042 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.962s user 0.015s sys 0.015s
I20260807 01:48:14.249598 27229 raft_consensus.cc:1240] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 6. Current term is 7. Ops: []
I20260807 01:48:14.239300 32074 raft_consensus.cc:697] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [term 10 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
W20260807 01:48:14.252316 32037 consensus_meta.cc:287] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.921s user 0.012s sys 0.006s
W20260807 01:48:14.251355 32147 log.cc:927] Time spent T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 1.687s user 0.013s sys 0.000s
W20260807 01:48:14.260015 27174 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:14.273999 27109 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.355s user 0.004s sys 0.035s
W20260807 01:48:14.291445 27109 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.372s user 0.004s sys 0.036s
I20260807 01:48:14.206285 32180 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:14.257157 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 352235us
W20260807 01:48:14.288532 27326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33976 (request call id 2662) took 1942 ms (client timeout 1952 ms). Trace:
I20260807 01:48:14.289300 27108 raft_consensus.cc:2468] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 7.
I20260807 01:48:14.245496 31756 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:48:14.293150 27326 rpcz_store.cc:269] 0807 01:48:12.329581 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:12.329752 (+ 171us) service_pool.cc:224] Handling call
0807 01:48:14.271377 (+1941625us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b540032b9d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:14.271670 (+ 293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":684682496}
I20260807 01:48:14.299618 27326 raft_consensus.cc:1275] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260807 01:48:14.300364 27326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59490 (request call id 2677) took 1284 ms. Trace:
I20260807 01:48:14.300688 27326 rpcz_store.cc:276] 0807 01:48:13.015581 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.298929 (+1283348us) service_pool.cc:224] Handling call
0807 01:48:14.299274 (+ 345us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:14.300059 (+ 785us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.300299 (+ 240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:14.253044 32069 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:14.244448 31948 raft_consensus.cc:515] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:14.269644 31117 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.175s user 0.028s sys 0.032s
I20260807 01:48:14.257866 32135 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:48:14.256496 32042 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.020s user 0.015s sys 0.015s
W20260807 01:48:14.338799 32115 log.cc:927] Time spent T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 2.234s user 0.007s sys 0.011s
W20260807 01:48:14.336935 32126 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.259s user 0.018s sys 0.048s
I20260807 01:48:14.297528 32037 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:14.337282 32108 raft_consensus.cc:493] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:48:14.351439 31117 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.251s user 0.029s sys 0.032s
I20260807 01:48:14.462946 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:14.463599 28709 raft_consensus.cc:3055] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:48:14.463869 28709 raft_consensus.cc:740] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:14.416678 31453 raft_consensus.cc:2804] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 01:48:14.434984 27338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33976 (request call id 2656) took 2866 ms. Trace:
I20260807 01:48:14.438850 31901 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:14.440649 32154 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:14.444579 32037 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:48:14.423957 32149 log.cc:927] Time spent T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 1.804s user 0.003s sys 0.012s
I20260807 01:48:14.453205 32130 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:14.416123 32043 raft_consensus.cc:493] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:14.424146 32074 consensus_queue.cc:237] T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:14.486536 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:14.458259 31943 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:14.458703 27328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33976 (request call id 2650) took 2891 ms. Trace:
I20260807 01:48:14.460309 31908 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:14.461145 27336 raft_consensus.cc:1240] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6. Ops: []
W20260807 01:48:14.420341 32126 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.338s user 0.018s sys 0.049s
I20260807 01:48:14.465685 27338 rpcz_store.cc:276] 0807 01:48:11.568848 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:11.759604 (+190756us) service_pool.cc:224] Handling call
0807 01:48:11.759959 (+ 355us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:12.663998 (+904039us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.434619 (+1770621us) spinlock_profiling.cc:243] Waited 889 ms on lock 0x7b5400165654. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:14.434868 (+ 249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2044824448}
I20260807 01:48:14.469484 27214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57662 (request call id 2804) took 2543 ms. Trace:
I20260807 01:48:14.474103 27227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41230 (request call id 591) took 1603 ms. Trace:
I20260807 01:48:14.474493 32174 leader_election.cc:290] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:14.474280 32158 log.cc:927] Time spent T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 1.732s user 0.012s sys 0.020s
I20260807 01:48:14.476204 32105 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:48:14.476456 32189 log.cc:927] Time spent T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.136s user 0.000s sys 0.001s
I20260807 01:48:14.477097 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:14.488287 27328 rpcz_store.cc:276] 0807 01:48:11.567040 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:11.754627 (+187587us) service_pool.cc:224] Handling call
0807 01:48:11.754985 (+ 358us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:11.755405 (+ 420us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:11.755435 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:11.755950 (+ 515us) log.cc:844] Serialized 26 byte log entry
0807 01:48:11.818464 (+ 62514us) raft_consensus.cc:1645] Marking committed up to 3
0807 01:48:11.818510 (+ 46us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:11.818587 (+ 77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:13.937242 (+2118655us) spinlock_profiling.cc:243] Waited 3.48 ms on lock 0x7b440002250c. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d5223e855 00007f7d5223f272 00007f7d54fc93bb 00007f7d54fc4c9f 00007f7d54fb8a5e 00007f7d54fb7da6 00007f7d54fb79c8 00007f7d54fc1411 00007f7d54fc13c9 00007f7d54fc1359
0807 01:48:14.458280 (+521038us) raft_consensus.cc:1685] finished
0807 01:48:14.458302 (+ 22us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:14.458638 (+ 336us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":95,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"spinlock_wait_cycles":7994240,"thread_start_us":61928,"threads_started":1,"wal-append.queue_time_us":124886}
I20260807 01:48:14.475723 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:14.464533 28709 consensus_queue.cc:260] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:14.514144 28709 raft_consensus.cc:3060] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:14.475817 32139 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:14.515347 27328 raft_consensus.cc:1275] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 01:48:14.496388 27336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59490 (request call id 2680) took 1467 ms. Trace:
I20260807 01:48:14.494999 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:14.467870 31453 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:14.497851 27227 rpcz_store.cc:276] 0807 01:48:12.870893 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:13.960746 (+1089853us) service_pool.cc:224] Handling call
0807 01:48:13.985983 (+ 25237us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:14.058796 (+ 72813us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:14.058827 (+ 31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:14.058873 (+ 46us) raft_consensus.cc:1645] Marking committed up to 7
0807 01:48:14.058909 (+ 36us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.473839 (+414930us) spinlock_profiling.cc:243] Waited 72.6 ms on lock 0x7b54000f2b54. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:14.473892 (+ 53us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:14.474037 (+ 145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166897536}
I20260807 01:48:14.467156 32108 raft_consensus.cc:515] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:14.497463 27214 rpcz_store.cc:276] 0807 01:48:11.918649 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:11.921149 (+ 2500us) service_pool.cc:224] Handling call
0807 01:48:11.921649 (+ 500us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:11.934676 (+ 13027us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:11.934709 (+ 33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:11.968664 (+ 33955us) log.cc:844] Serialized 26 byte log entry
0807 01:48:12.418624 (+449960us) raft_consensus.cc:1645] Marking committed up to 2
0807 01:48:12.418667 (+ 43us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:12.418745 (+ 78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:14.232834 (+1814089us) spinlock_profiling.cc:243] Waited 4.7 ms on lock 0x7b440002250c. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d5223e855 00007f7d5223f272 00007f7d54fc93bb 00007f7d54fc4c9f 00007f7d54fb8a5e 00007f7d54fb7da6 00007f7d54fb79c8 00007f7d54fc1411 00007f7d54fc13c9 00007f7d54fc1359
0807 01:48:14.461209 (+228375us) raft_consensus.cc:1685] finished
0807 01:48:14.461232 (+ 23us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:14.461525 (+ 293us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13629,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":12426,"spinlock_wait_cycles":10810624,"thread_start_us":356180,"threads_started":1,"wal-append.queue_time_us":476153}
I20260807 01:48:14.469990 31898 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:14.470789 32110 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:14.507769 27338 raft_consensus.cc:1240] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 8. Current term is 9. Ops: []
I20260807 01:48:14.523576 27338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59490 (request call id 2681) took 1493 ms. Trace:
I20260807 01:48:14.518246 27328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59490 (request call id 2687) took 1486 ms. Trace:
I20260807 01:48:14.524298 27338 rpcz_store.cc:276] 0807 01:48:13.029652 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.507103 (+1477451us) service_pool.cc:224] Handling call
0807 01:48:14.507549 (+ 446us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:14.522895 (+ 15346us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.523454 (+ 559us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:14.528384 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:14.525797 27328 rpcz_store.cc:276] 0807 01:48:13.031471 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.511337 (+1479866us) service_pool.cc:224] Handling call
0807 01:48:14.511657 (+ 320us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:14.515766 (+ 4109us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.518141 (+ 2375us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:14.516690 27336 rpcz_store.cc:276] 0807 01:48:13.029388 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.460641 (+1431253us) service_pool.cc:224] Handling call
0807 01:48:14.460950 (+ 309us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:14.496019 (+ 35069us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.496312 (+ 293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:14.529829 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:14.508853 31948 leader_election.cc:290] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:14.521150 31453 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:14.478976 31901 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:14.485260 32154 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:14.489847 32043 raft_consensus.cc:515] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:14.557701 28711 raft_consensus.cc:2410] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260807 01:48:14.565019 28715 raft_consensus.cc:1240] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 6. Current term is 7. Ops: []
I20260807 01:48:14.567708 27325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33976 (request call id 2658) took 2253 ms. Trace:
I20260807 01:48:14.560302 27336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36604 (request call id 843) took 1516 ms. Trace:
I20260807 01:48:14.589534 27336 rpcz_store.cc:276] 0807 01:48:13.043819 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.531762 (+1487943us) service_pool.cc:224] Handling call
0807 01:48:14.559404 (+ 27642us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:14.559926 (+ 522us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:14.559953 (+ 27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:14.559997 (+ 44us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:48:14.560029 (+ 32us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.560086 (+ 57us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:14.560235 (+ 149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:14.587795 27325 rpcz_store.cc:276] 0807 01:48:12.314645 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:12.317077 (+ 2432us) service_pool.cc:224] Handling call
0807 01:48:12.317393 (+ 316us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:12.327051 (+ 9658us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:12.327079 (+ 28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:12.327570 (+ 491us) log.cc:844] Serialized 26 byte log entry
0807 01:48:12.472182 (+144612us) raft_consensus.cc:1645] Marking committed up to 2
0807 01:48:12.472242 (+ 60us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:12.472317 (+ 75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:14.214752 (+1742435us) spinlock_profiling.cc:243] Waited 6.64 ms on lock 0x7b440002250c. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d5223e855 00007f7d5223f272 00007f7d54fc93bb 00007f7d54fc4c9f 00007f7d54fb8a5e 00007f7d54fb7da6 00007f7d54fb79c8 00007f7d54fc1411 00007f7d54fc13c9 00007f7d54fc1359
0807 01:48:14.562492 (+347740us) raft_consensus.cc:1685] finished
0807 01:48:14.562515 (+ 23us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:14.567614 (+ 5099us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9288,"raft.queue_time_us":29999,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":51310,"spinlock_wait_cycles":15262592,"thread_start_us":63029,"threads_started":1,"wal-append.queue_time_us":205090}
I20260807 01:48:14.599704 27325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36604 (request call id 844) took 1508 ms. Trace:
I20260807 01:48:14.600271 27325 rpcz_store.cc:276] 0807 01:48:13.091875 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.597130 (+1505255us) service_pool.cc:224] Handling call
0807 01:48:14.597591 (+ 461us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:14.599327 (+ 1736us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:14.599355 (+ 28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:14.599399 (+ 44us) raft_consensus.cc:1645] Marking committed up to 7
0807 01:48:14.599440 (+ 41us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.599496 (+ 56us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:14.599624 (+ 128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:14.604568 27336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33976 (request call id 2670) took 1432 ms. Trace:
I20260807 01:48:14.612002 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:14.606451 27325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36604 (request call id 846) took 1376 ms. Trace:
I20260807 01:48:14.615994 27325 rpcz_store.cc:276] 0807 01:48:13.230065 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.605579 (+1375514us) service_pool.cc:224] Handling call
0807 01:48:14.605924 (+ 345us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:14.606113 (+ 189us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:14.606137 (+ 24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:14.606199 (+ 62us) raft_consensus.cc:1645] Marking committed up to 7
0807 01:48:14.606226 (+ 27us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.606276 (+ 50us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:14.606385 (+ 109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:14.617293 27336 rpcz_store.cc:276] 0807 01:48:13.172610 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.603595 (+1430985us) service_pool.cc:224] Handling call
0807 01:48:14.603988 (+ 393us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:14.604212 (+ 224us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:14.604238 (+ 26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:14.604282 (+ 44us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:48:14.604324 (+ 42us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.604377 (+ 53us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:14.604498 (+ 121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:14.620200 28709 raft_consensus.cc:2468] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:48:14.627051 27336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36604 (request call id 847) took 1292 ms. Trace:
I20260807 01:48:14.633386 27336 rpcz_store.cc:276] 0807 01:48:13.334701 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.621951 (+1287250us) service_pool.cc:224] Handling call
0807 01:48:14.622318 (+ 367us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:14.626584 (+ 4266us) raft_consensus.cc:1534] Early marking committed up to index 7
0807 01:48:14.626621 (+ 37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:14.626686 (+ 65us) raft_consensus.cc:1645] Marking committed up to 7
0807 01:48:14.626728 (+ 42us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.626816 (+ 88us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:14.626977 (+ 161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2947}
W20260807 01:48:14.630820 28707 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.785s user 0.006s sys 0.042s
W20260807 01:48:14.650331 28707 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.805s user 0.009s sys 0.042s
I20260807 01:48:14.655102 32153 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
W20260807 01:48:14.610823 31509 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.639s user 0.035s sys 0.040s
W20260807 01:48:14.629805 28660 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.912s user 0.024s sys 0.039s
W20260807 01:48:14.650682 27327 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.810s user 0.012s sys 0.029s
I20260807 01:48:14.567812 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:14.652554 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:14.679930 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:14.650142 32151 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.726s user 0.035s sys 0.036s
I20260807 01:48:14.557076 31898 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:14.658465 27218 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.781s user 0.008s sys 0.033s
I20260807 01:48:14.557291 32110 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:14.670274 28660 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.952s user 0.024s sys 0.045s
W20260807 01:48:14.704181 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 986393us
W20260807 01:48:14.670344 27327 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.829s user 0.012s sys 0.030s
W20260807 01:48:14.670727 31509 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.690s user 0.036s sys 0.040s
W20260807 01:48:14.655855 32163 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.304s user 0.008s sys 0.034s
W20260807 01:48:14.696275 27218 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.819s user 0.009s sys 0.034s
W20260807 01:48:14.726688 27337 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s user 0.025s sys 0.038s
W20260807 01:48:14.690095 32039 consensus_meta.cc:287] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.838s user 0.012s sys 0.006s
I20260807 01:48:14.672245 31901 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:14.733460 27216 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.707s user 0.010s sys 0.022s
W20260807 01:48:14.737112 27216 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.711s user 0.010s sys 0.023s
W20260807 01:48:14.733875 27337 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.521s user 0.025s sys 0.038s
W20260807 01:48:14.691813 32151 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.777s user 0.036s sys 0.036s
W20260807 01:48:14.691082 31954 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.963s user 0.019s sys 0.045s
I20260807 01:48:14.738425 27337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36604 (request call id 842) took 1794 ms. Trace:
W20260807 01:48:14.733240 32163 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.387s user 0.008s sys 0.037s
I20260807 01:48:14.742056 28702 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:14.739153 32039 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:14.748950 27226 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s user 0.011s sys 0.013s
I20260807 01:48:14.735347 32139 raft_consensus.cc:493] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:14.739841 32202 log.cc:927] Time spent T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.061s user 0.000s sys 0.000s
W20260807 01:48:14.743033 31954 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.015s user 0.020s sys 0.045s
W20260807 01:48:14.753023 28660 connection.cc:256] RPC call timeout handler was delayed by -1.0684059110935777s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1874s for a response
W20260807 01:48:14.762277 27175 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s user 0.000s sys 0.025s
I20260807 01:48:14.763403 28660 leader_election.cc:304] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
W20260807 01:48:14.751536 27226 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s user 0.011s sys 0.013s
I20260807 01:48:14.743854 27337 rpcz_store.cc:276] 0807 01:48:12.943556 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.202097 (+1258541us) service_pool.cc:224] Handling call
0807 01:48:14.212091 (+ 9994us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:14.212316 (+ 225us) raft_consensus.cc:1534] Early marking committed up to index 6
0807 01:48:14.212960 (+ 644us) log.cc:844] Serialized 14 byte log entry
0807 01:48:14.738021 (+525061us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:14.738083 (+ 62us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:48:14.738128 (+ 45us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.738220 (+ 92us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:14.738357 (+ 137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1839,"prepare.run_cpu_time_us":156,"prepare.run_wall_time_us":3813,"raft.queue_time_us":234,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"thread_start_us":524664,"threads_started":1}
W20260807 01:48:14.752928 27172 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.638s user 0.015s sys 0.028s
I20260807 01:48:14.751495 27101 raft_consensus.cc:1240] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6. Ops: []
I20260807 01:48:14.734048 32140 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:14.737900 32174 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:14.668537 32153 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:14.733631 32141 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:48:14.762658 27175 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s user 0.000s sys 0.025s
W20260807 01:48:14.749580 27170 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s user 0.014s sys 0.008s
W20260807 01:48:14.778173 27170 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.637s user 0.015s sys 0.008s
W20260807 01:48:14.776479 27172 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.662s user 0.015s sys 0.030s
W20260807 01:48:14.779165 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 667987us
W20260807 01:48:14.778674 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:14.779044 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:14.781800 28660 leader_election.cc:400] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for earlier term 5. Current term is 6.
I20260807 01:48:14.782337 28660 leader_election.cc:403] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260807 01:48:14.778589 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 648991us
I20260807 01:48:14.783331 28660 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: fcf4e45defb94276bcfa68392711dade
W20260807 01:48:14.776784 27104 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.854s user 0.011s sys 0.022s
W20260807 01:48:14.758683 31203 consensus_meta.cc:287] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.572s user 0.007s sys 0.007s
W20260807 01:48:14.787107 27104 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.864s user 0.011s sys 0.022s
I20260807 01:48:14.791555 27105 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 3094) took 1710 ms. Trace:
W20260807 01:48:14.792681 27170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:14.793256 28702 raft_consensus.cc:2410] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260807 01:48:14.789395 32126 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:14.796772 31117 raft_consensus.cc:2804] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:48:14.800005 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:14.800932 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:14.802044 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:14.786855 31954 raft_consensus.cc:2804] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:48:14.793519 31752 raft_consensus.cc:2749] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for earlier term 5. Current term is 6.
I20260807 01:48:14.803897 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:14.804303 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:14.804591 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:14.792017 27105 rpcz_store.cc:276] 0807 01:48:13.080897 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:13.081046 (+ 149us) service_pool.cc:224] Handling call
0807 01:48:13.081439 (+ 393us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:13.304614 (+223175us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.791246 (+1486632us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b5400170554. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:14.791487 (+ 241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":511588608}
I20260807 01:48:14.800819 32154 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:14.794215 27172 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
W20260807 01:48:14.764616 31896 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.625s user 0.012s sys 0.014s
I20260807 01:48:14.820035 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:14.823186 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:14.760851 32139 raft_consensus.cc:515] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:14.826696 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:14.827957 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:48:14.828406 27109 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s user 0.004s sys 0.016s
I20260807 01:48:14.828547 27214 raft_consensus.cc:2468] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
W20260807 01:48:14.814815 32194 log.cc:927] Time spent T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.162s user 0.000s sys 0.001s
I20260807 01:48:14.812891 31117 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:14.782039 32174 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:14.840574 27327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36604 (request call id 838) took 2037 ms. Trace:
W20260807 01:48:14.831749 27109 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.539s user 0.004s sys 0.016s
I20260807 01:48:14.831753 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:14.845549 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:48:14.824424 31896 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.685s user 0.013s sys 0.014s
I20260807 01:48:14.846221 27105 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Advancing to term 10
W20260807 01:48:14.831655 32052 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.791s user 0.016s sys 0.029s
I20260807 01:48:14.846728 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:14.832453 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:14.848268 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:14.814834 31954 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:14.822392 28713 raft_consensus.cc:3060] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:14.834597 27172 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:14.804561 31203 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:14.788914 32140 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:14.789510 32141 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:14.853149 27286 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s user 0.000s sys 0.021s
I20260807 01:48:14.824167 32135 raft_consensus.cc:2749] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260807 01:48:14.845342 27109 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1795) took 1988 ms. Trace:
I20260807 01:48:14.844261 27327 rpcz_store.cc:276] 0807 01:48:12.802872 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:12.804113 (+ 1241us) service_pool.cc:224] Handling call
0807 01:48:12.806693 (+ 2580us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:12.807097 (+ 404us) raft_consensus.cc:1534] Early marking committed up to index 8
0807 01:48:13.787832 (+980735us) log.cc:844] Serialized 14 byte log entry
0807 01:48:14.731463 (+943631us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:14.733082 (+ 1619us) log.cc:844] Serialized 26 byte log entry
0807 01:48:14.733279 (+ 197us) raft_consensus.cc:1645] Marking committed up to 8
0807 01:48:14.733319 (+ 40us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.733413 (+ 94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:14.840344 (+106931us) raft_consensus.cc:1685] finished
0807 01:48:14.840365 (+ 21us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:14.840507 (+ 142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20029,"prepare.queue_time_us":354714,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":28336,"raft.run_cpu_time_us":350,"raft.run_wall_time_us":10837,"thread_start_us":1817282,"threads_started":2,"wal-append.queue_time_us":1045038}
I20260807 01:48:14.814783 32126 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:14.850431 32052 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.809s user 0.016s sys 0.029s
W20260807 01:48:14.861850 27286 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s user 0.000s sys 0.021s
I20260807 01:48:14.843160 31117 raft_consensus.cc:3060] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:14.863077 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:14.926421 27214 raft_consensus.cc:2393] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 9: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:14.863658 27109 rpcz_store.cc:276] 0807 01:48:12.856798 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:12.883600 (+ 26802us) service_pool.cc:224] Handling call
0807 01:48:12.886770 (+ 3170us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:12.886974 (+ 204us) raft_consensus.cc:1534] Early marking committed up to index 4
0807 01:48:14.292148 (+1405174us) log.cc:844] Serialized 14 byte log entry
0807 01:48:14.844914 (+552766us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:14.844970 (+ 56us) raft_consensus.cc:1645] Marking committed up to 4
0807 01:48:14.845019 (+ 49us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.845085 (+ 66us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:14.845275 (+ 190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":41965,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":260,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":6541,"thread_start_us":1924868,"threads_started":2}
I20260807 01:48:14.866501 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:14.853098 31954 raft_consensus.cc:3060] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:14.872720 31716 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:14.860776 31901 raft_consensus.cc:2749] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:14.875509 32110 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:14.876714 27218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57662 (request call id 2806) took 2957 ms. Trace:
I20260807 01:48:14.877130 32186 raft_consensus.cc:3055] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Stepping down as leader of term 6
I20260807 01:48:14.879405 26952 catalog_manager.cc:5697] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:48:14.884179 31953 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:14.885532 32043 leader_election.cc:290] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:14.890316 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:14.842962 32108 leader_election.cc:290] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:14.895447 31631 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:14.898411 28710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48350 (request call id 922) took 2733 ms. Trace:
I20260807 01:48:14.904575 28706 raft_consensus.cc:1275] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260807 01:48:14.837567 31752 consensus_queue.cc:1048] T aff03a52fbfd498ab6d04a579e96fd0e P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260807 01:48:14.996546 26952 catalog_manager.cc:5697] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:48:14.909785 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:14.940351 27100 raft_consensus.cc:1240] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 9 LEADER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 8. Current term is 9. Ops: []
I20260807 01:48:14.909065 31509 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:14.945982 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:14.944208 27174 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s user 0.013s sys 0.021s
I20260807 01:48:14.957538 27331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59490 (request call id 2679) took 1928 ms. Trace:
W20260807 01:48:14.924551 31756 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.531s user 0.012s sys 0.016s
W20260807 01:48:14.957741 32053 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.038s user 0.032s sys 0.020s
W20260807 01:48:14.962603 27102 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41080 (request call id 3079) took 3352 ms (client timeout 1703 ms). Trace:
I20260807 01:48:14.924485 31743 raft_consensus.cc:493] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:14.958711 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:14.959332 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:48:14.988467 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 428970us
I20260807 01:48:14.966445 27218 rpcz_store.cc:276] 0807 01:48:11.919268 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:11.922018 (+ 2750us) service_pool.cc:224] Handling call
0807 01:48:11.953778 (+ 31760us) raft_consensus.cc:1471] Updating replica for 2 ops
0807 01:48:11.963836 (+ 10058us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:11.963865 (+ 29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0807 01:48:11.989023 (+ 25158us) log.cc:844] Serialized 52 byte log entry
0807 01:48:13.877475 (+1888452us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b180001b9a0. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d5b233f31 00007f7d5b2406d1 00007f7d522fa7b7 00007f7d5b1a2fc9 00007f7d52316fdb 00007f7d523152db 00007f7d52317542 00007f7d54fb73fb 00007f7d54fbcf13 00007f7d54fbd2c1 00007f7d553d9def
0807 01:48:14.731996 (+854521us) raft_consensus.cc:1645] Marking committed up to 3
0807 01:48:14.732050 (+ 54us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.732141 (+ 91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:14.876329 (+144188us) raft_consensus.cc:1685] finished
0807 01:48:14.876352 (+ 23us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:14.876639 (+ 287us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4258,"raft.queue_time_us":23764,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":8760,"spinlock_wait_cycles":55886592,"thread_start_us":854221,"threads_started":1,"wal-append.queue_time_us":2859964}
I20260807 01:48:14.990948 28710 rpcz_store.cc:276] 0807 01:48:12.159295 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:12.159546 (+ 251us) service_pool.cc:224] Handling call
0807 01:48:12.160195 (+ 649us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:13.729390 (+1569195us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.877936 (+1148546us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b54004f3654. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:14.892839 (+ 14903us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3596059392}
I20260807 01:48:14.936729 32179 raft_consensus.cc:493] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:14.993571 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:14.907727 27105 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 10.
I20260807 01:48:14.942344 31948 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:14.924039 32142 consensus_queue.cc:1059] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260807 01:48:14.924443 32135 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:14.868505 31898 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:15.255497 28714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54eb75662df043488517f09bff1dfcff"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:34614
I20260807 01:48:15.256652 28714 raft_consensus.cc:606] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 9 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:48:15.110733 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:15.259563 27216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41230 (request call id 594) took 2341 ms. Trace:
W20260807 01:48:15.052650 27110 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 1793) took 2196 ms (client timeout 2905 ms). Trace:
I20260807 01:48:15.026816 27336 raft_consensus.cc:1240] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 9. Current term is 10. Ops: []
I20260807 01:48:15.025295 27101 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 3100) took 1938 ms. Trace:
W20260807 01:48:15.264441 27102 rpcz_store.cc:269] 0807 01:48:11.610368 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:11.615738 (+ 5370us) service_pool.cc:224] Handling call
0807 01:48:14.962363 (+3346625us) spinlock_profiling.cc:243] Waited 2.26 s on lock 0x7b5400336dd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:14.962495 (+ 132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5194372480}
I20260807 01:48:14.963241 32105 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:14.995268 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:15.262109 27105 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 1805) took 1139 ms. Trace:
I20260807 01:48:15.263969 27101 rpcz_store.cc:276] 0807 01:48:13.086762 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:13.725131 (+638369us) service_pool.cc:224] Handling call
0807 01:48:13.748199 (+ 23068us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:14.776547 (+1028348us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:15.024979 (+248432us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54001720d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:15.025216 (+ 237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2306787712}
I20260807 01:48:15.285648 27105 rpcz_store.cc:276] 0807 01:48:14.122723 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.845152 (+722429us) service_pool.cc:224] Handling call
0807 01:48:15.262043 (+416891us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 01:48:15.266077 27110 rpcz_store.cc:269] 0807 01:48:12.856170 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:12.856300 (+ 130us) service_pool.cc:224] Handling call
0807 01:48:15.052306 (+2196006us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b540024efd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:15.052546 (+ 240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":563751296}
I20260807 01:48:14.988791 32166 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:48:15.278278 27174 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.986s user 0.021s sys 0.047s
I20260807 01:48:15.282826 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:14.983726 32186 raft_consensus.cc:740] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:15.260076 27216 rpcz_store.cc:276] 0807 01:48:12.918319 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:13.978515 (+1060196us) service_pool.cc:224] Handling call
0807 01:48:14.024188 (+ 45673us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:14.024635 (+ 447us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:14.024666 (+ 31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:14.025942 (+ 1276us) log.cc:844] Serialized 26 byte log entry
0807 01:48:14.737533 (+711591us) raft_consensus.cc:1645] Marking committed up to 9
0807 01:48:14.737576 (+ 43us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.737644 (+ 68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:15.258320 (+520676us) raft_consensus.cc:1685] finished
0807 01:48:15.258632 (+ 312us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:15.259427 (+ 795us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":209,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":16026,"thread_start_us":711140,"threads_started":1,"wal-append.queue_time_us":976673}
I20260807 01:48:15.287786 28708 raft_consensus.cc:2393] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 6: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:15.267623 26952 catalog_manager.cc:5697] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "57bb4985157447f0a3ab680fa0026f12" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:48:15.018378 32108 raft_consensus.cc:493] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:15.001820 31943 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:15.292194 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:15.023047 32201 log.cc:927] Time spent T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.056s user 0.000s sys 0.000s
W20260807 01:48:15.269169 32053 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.345s user 0.035s sys 0.031s
I20260807 01:48:15.294092 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:15.294689 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:15.276822 27331 rpcz_store.cc:276] 0807 01:48:13.029053 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.457811 (+1428758us) service_pool.cc:224] Handling call
0807 01:48:14.458182 (+ 371us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:14.458423 (+ 241us) raft_consensus.cc:1534] Early marking committed up to index 6
0807 01:48:14.458758 (+ 335us) log.cc:844] Serialized 14 byte log entry
0807 01:48:14.942829 (+484071us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:14.957189 (+ 14360us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:48:14.957250 (+ 61us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.957331 (+ 81us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:14.957468 (+ 137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7840,"prepare.queue_time_us":9099,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":24074,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":13368,"thread_start_us":475812,"threads_started":1}
I20260807 01:48:15.328230 27105 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 1808) took 1196 ms. Trace:
I20260807 01:48:15.328450 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:15.328604 27105 rpcz_store.cc:276] 0807 01:48:14.131592 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:15.293257 (+1161665us) service_pool.cc:224] Handling call
0807 01:48:15.328169 (+ 34912us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 01:48:15.299005 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:15.299872 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:48:15.329631 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:15.265798 31756 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.903s user 0.014s sys 0.025s
I20260807 01:48:15.300988 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:15.309088 26952 catalog_manager.cc:5697] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:48:15.214140 32154 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:15.272930 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:15.017920 32071 raft_consensus.cc:2749] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260807 01:48:15.314090 27328 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s user 0.005s sys 0.021s
I20260807 01:48:15.289757 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:15.322304 27110 raft_consensus.cc:2410] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260807 01:48:15.274479 32039 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:15.330070 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:15.332383 27328 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.662s user 0.006s sys 0.021s
I20260807 01:48:15.328966 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:15.337415 27110 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 1800) took 1218 ms. Trace:
I20260807 01:48:15.338002 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:15.017417 31631 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:15.335566 27331 raft_consensus.cc:2393] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 8: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:15.337558 27327 raft_consensus.cc:2393] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 6: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:15.357035 27214 raft_consensus.cc:3060] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
W20260807 01:48:15.314138 32130 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.516s user 0.009s sys 0.015s
I20260807 01:48:15.358923 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:15.367468 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:15.368297 27218 raft_consensus.cc:2393] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 5: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:15.284319 31509 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:15.273895 31948 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:15.362702 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:15.374076 27105 raft_consensus.cc:3060] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:15.331998 26952 catalog_manager.cc:5697] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 4 to 5, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 5 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } health_report { overall_health: HEALTHY } } }
I20260807 01:48:15.376549 26952 catalog_manager.cc:5697] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 6 to 7, leader changed from 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) to 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1). New cstate: current_term: 7 leader_uuid: "3262a9d183914b699227fdba0ca5bed3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } } }
I20260807 01:48:15.364696 27110 rpcz_store.cc:276] 0807 01:48:14.118843 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:15.291894 (+1173051us) service_pool.cc:224] Handling call
0807 01:48:15.337336 (+ 45442us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:15.284134 31743 raft_consensus.cc:515] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:15.274798 32135 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:15.324179 32153 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:15.370806 27216 raft_consensus.cc:2393] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 5: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:15.384721 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:15.388432 27218 raft_consensus.cc:3060] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
W20260807 01:48:15.367063 32130 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.569s user 0.009s sys 0.016s
I20260807 01:48:15.383764 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:15.394556 27216 raft_consensus.cc:3060] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:15.325874 32108 raft_consensus.cc:515] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:15.331719 32139 leader_election.cc:290] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:15.276584 32179 raft_consensus.cc:515] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:15.378840 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:15.402072 27110 raft_consensus.cc:3060] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:15.363335 31203 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:15.404898 27105 raft_consensus.cc:2468] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 7.
I20260807 01:48:15.404639 27175 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
W20260807 01:48:15.381152 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 455395us
W20260807 01:48:15.429344 28660 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s user 0.026s sys 0.050s
I20260807 01:48:15.372337 27327 raft_consensus.cc:1240] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 4. Current term is 5. Ops: []
W20260807 01:48:15.429940 28660 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s user 0.026s sys 0.050s
I20260807 01:48:15.331403 31943 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:15.378098 26952 catalog_manager.cc:5697] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 3 to 4, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 4 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:48:15.433593 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:15.409641 32229 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:15.434657 26952 catalog_manager.cc:5697] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:48:15.357421 32040 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:15.301525 32105 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: NON_VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: true } }
I20260807 01:48:15.410789 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:15.425491 27105 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 1804) took 1305 ms. Trace:
I20260807 01:48:15.359484 32154 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:15.360381 27101 raft_consensus.cc:2410] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 8 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260807 01:48:15.437354 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 191022us
W20260807 01:48:15.429998 27338 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s user 0.021s sys 0.022s
I20260807 01:48:15.427155 27214 raft_consensus.cc:2468] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 7.
I20260807 01:48:15.427369 27287 leader_election.cc:304] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:15.426564 27218 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:48:15.416102 32126 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:15.438795 27331 raft_consensus.cc:3060] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:15.403895 32131 consensus_queue.cc:1059] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260807 01:48:15.450261 27105 rpcz_store.cc:276] 0807 01:48:14.119924 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:15.346493 (+1226569us) service_pool.cc:224] Handling call
0807 01:48:15.425410 (+ 78917us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 01:48:15.450307 27338 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s user 0.023s sys 0.022s
I20260807 01:48:15.451187 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:15.451179 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:15.471390 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
W20260807 01:48:15.431419 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 624027us
I20260807 01:48:15.439412 27101 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 1807) took 1308 ms. Trace:
I20260807 01:48:15.369966 32186 consensus_queue.cc:260] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:15.433277 32206 log.cc:927] Time spent T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.430s user 0.000s sys 0.001s
I20260807 01:48:15.509145 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:15.436298 26952 catalog_manager.cc:5697] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:48:15.474419 27101 rpcz_store.cc:276] 0807 01:48:14.131343 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:15.295011 (+1163668us) service_pool.cc:224] Handling call
0807 01:48:15.439346 (+144335us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 01:48:15.432977 32209 log.cc:927] Time spent T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.473s user 0.000s sys 0.001s
I20260807 01:48:15.492273 27110 raft_consensus.cc:2468] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
W20260807 01:48:15.496747 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 205404us
I20260807 01:48:15.498567 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:15.500557 27229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41230 (request call id 587) took 2662 ms. Trace:
W20260807 01:48:15.526968 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 52215.1us
I20260807 01:48:15.531023 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:15.532244 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:15.462321 32224 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:15.510260 27327 raft_consensus.cc:3060] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:15.510738 26952 catalog_manager.cc:5697] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 2 to 3, leader changed from fcf4e45defb94276bcfa68392711dade (127.24.4.3) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 3 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: UNKNOWN } } }
I20260807 01:48:15.581058 26952 catalog_manager.cc:5697] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 2 to 4, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 4 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:48:15.507747 27331 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
I20260807 01:48:15.583688 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:15.584539 27331 raft_consensus.cc:2393] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 7: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:15.586416 27214 raft_consensus.cc:1240] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260807 01:48:15.590288 27212 raft_consensus.cc:1240] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 8. Current term is 9. Ops: []
I20260807 01:48:15.439284 31953 consensus_queue.cc:1048] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.093s
W20260807 01:48:15.571971 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 101681us
I20260807 01:48:15.531798 27229 rpcz_store.cc:276] 0807 01:48:12.838417 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:13.925134 (+1086717us) service_pool.cc:224] Handling call
0807 01:48:13.952297 (+ 27163us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:14.257684 (+305387us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:15.500240 (+1242556us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b5400264dd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:15.500481 (+ 241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":683225856}
I20260807 01:48:15.601279 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:15.437186 32229 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:15.578500 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:15.605790 27327 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:48:15.607892 31631 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:48:15.608523 32105 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:15.609098 32105 consensus_queue.cc:1048] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:48:15.604482 27230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41230 (request call id 584) took 2802 ms. Trace:
I20260807 01:48:15.596088 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:15.435153 32220 raft_consensus.cc:2749] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 01:48:15.601863 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
W20260807 01:48:15.574606 32216 log.cc:927] Time spent T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.380s user 0.001s sys 0.001s
I20260807 01:48:15.623286 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:15.623950 28715 raft_consensus.cc:2376] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5. Re-sending same reply.
I20260807 01:48:15.599591 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:15.531025 31896 raft_consensus.cc:3055] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:48:15.465368 32040 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:15.615232 27110 raft_consensus.cc:2468] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:48:15.609913 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:15.624825 28708 raft_consensus.cc:2468] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:48:15.586424 32254 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:48:15.618940 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 162533us
I20260807 01:48:15.590065 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:48:15.519675 28713 consensus_meta.cc:287] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 0.651s user 0.007s sys 0.003s
I20260807 01:48:15.591552 26952 catalog_manager.cc:5697] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f reported cstate change: term changed from 2 to 3, leader changed from 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1) to 3478729505b046c2a0051c2d0130873f (127.24.4.2). New cstate: current_term: 3 leader_uuid: "3478729505b046c2a0051c2d0130873f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 01:48:15.532567 27105 raft_consensus.cc:2393] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 6: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
W20260807 01:48:15.620024 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 182025us
I20260807 01:48:15.621011 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
W20260807 01:48:15.615304 32228 log.cc:927] Time spent T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.303s user 0.001s sys 0.000s
I20260807 01:48:15.623476 27212 raft_consensus.cc:3060] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
W20260807 01:48:15.618311 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 44731.6us
W20260807 01:48:15.616420 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 161775us
I20260807 01:48:15.592971 32153 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
W20260807 01:48:15.615540 32212 log.cc:927] Time spent T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.305s user 0.000s sys 0.001s
I20260807 01:48:15.517434 32139 consensus_queue.cc:1059] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.093s
I20260807 01:48:15.656348 27099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 3098) took 2570 ms. Trace:
I20260807 01:48:15.656769 27099 rpcz_store.cc:276] 0807 01:48:13.086244 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:13.716355 (+630111us) service_pool.cc:224] Handling call
0807 01:48:13.716715 (+ 360us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:13.943229 (+226514us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:15.656106 (+1712877us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b5400076554. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:15.656287 (+ 181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":519182208}
I20260807 01:48:15.614435 32043 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:48:15.636585 28713 raft_consensus.cc:2468] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:15.626111 31896 raft_consensus.cc:740] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:15.660378 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:15.600939 32249 raft_consensus.cc:2804] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:48:15.585500 32126 consensus_queue.cc:1059] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260807 01:48:15.643090 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:15.649353 27172 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:15.649554 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:15.614705 31631 consensus_queue.cc:1048] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260807 01:48:15.679529 28702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48362 (request call id 850) took 1218 ms. Trace:
I20260807 01:48:15.679957 28702 rpcz_store.cc:276] 0807 01:48:14.461377 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.465148 (+ 3771us) service_pool.cc:224] Handling call
0807 01:48:15.679299 (+1214151us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b5400721ad4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:15.679458 (+ 159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":529558272}
I20260807 01:48:15.654346 27212 raft_consensus.cc:2468] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
I20260807 01:48:15.614475 27230 rpcz_store.cc:276] 0807 01:48:12.802055 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:13.918022 (+1115967us) service_pool.cc:224] Handling call
0807 01:48:15.531880 (+1613858us) spinlock_profiling.cc:243] Waited 1.03 ms on lock 0x7b5800135398. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e0521 00007f7d522e03d2 00007f7d5b24c219 00007f7d5a39f640 00007f7d5a467ebb 00007f7d5a467f41 00007f7d5a429dd0 00007f7d5a414739 00007f7d5a4141e2 00007f7d53959d34
0807 01:48:15.576248 (+ 44368us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:15.604077 (+ 27829us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:15.604118 (+ 41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:15.604172 (+ 54us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:48:15.604209 (+ 37us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:15.604281 (+ 72us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:15.604413 (+ 132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18644,"raft.queue_time_us":18765,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":15287,"spinlock_wait_cycles":2358016}
W20260807 01:48:15.682523 32238 log.cc:927] Time spent T 73e9ae0228aa41a996315efe07c04299 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.208s user 0.000s sys 0.001s
I20260807 01:48:15.684269 32040 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:15.657929 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:15.687597 27099 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 8.
I20260807 01:48:15.687834 27216 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
I20260807 01:48:15.636868 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:15.662705 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:15.664731 27229 raft_consensus.cc:2468] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
W20260807 01:48:15.665381 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:15.688910 27331 raft_consensus.cc:3060] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
W20260807 01:48:15.647632 31954 consensus_meta.cc:287] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 0.716s user 0.012s sys 0.000s
I20260807 01:48:15.668877 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:15.671046 27110 raft_consensus.cc:3060] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:15.642465 28655 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
W20260807 01:48:15.658171 32234 log.cc:927] Time spent T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.286s user 0.000s sys 0.001s
I20260807 01:48:15.718407 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:15.719197 27327 raft_consensus.cc:2468] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
W20260807 01:48:15.720289 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 71011.8us
I20260807 01:48:15.670475 27105 raft_consensus.cc:3060] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:15.674662 32229 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:15.655880 31948 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:48:15.681143 31453 consensus_meta.cc:287] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 1.083s user 0.004s sys 0.007s
W20260807 01:48:15.676337 27108 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 1798) took 1793 ms (client timeout 1955 ms). Trace:
I20260807 01:48:15.678520 32071 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
W20260807 01:48:15.678831 32246 log.cc:927] Time spent T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.079s user 0.000s sys 0.000s
I20260807 01:48:15.594765 32224 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:15.716523 32235 log.cc:927] Time spent T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.263s user 0.000s sys 0.001s
I20260807 01:48:15.702965 27101 raft_consensus.cc:2393] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 7: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:15.703994 27331 raft_consensus.cc:2468] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
I20260807 01:48:15.755165 27105 raft_consensus.cc:2468] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 8.
I20260807 01:48:15.755528 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:15.755990 27105 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52556 (request call id 711) took 1016 ms. Trace:
I20260807 01:48:15.756192 27331 raft_consensus.cc:2468] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 8.
I20260807 01:48:15.756516 27105 rpcz_store.cc:276] 0807 01:48:14.739299 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:15.642817 (+903518us) service_pool.cc:224] Handling call
0807 01:48:15.755890 (+113073us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:15.757085 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:15.757488 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:15.637784 32254 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:15.667377 32043 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:15.670343 31509 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:15.659137 32153 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:15.732426 31953 raft_consensus.cc:2749] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260807 01:48:15.666870 32231 log.cc:927] Time spent T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.296s user 0.001s sys 0.000s
I20260807 01:48:15.711158 31896 consensus_queue.cc:260] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:15.714704 32241 log.cc:927] Time spent T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.142s user 0.000s sys 0.000s
I20260807 01:48:15.655628 32166 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:48:15.713933 28707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48362 (request call id 845) took 3752 ms. Trace:
I20260807 01:48:15.663496 31652 raft_consensus.cc:493] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:15.672010 32232 log.cc:927] Time spent T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.293s user 0.000s sys 0.001s
W20260807 01:48:15.730233 27213 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57662 (request call id 2817) took 2639 ms (client timeout 2082 ms). Trace:
I20260807 01:48:15.675143 32151 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:15.731501 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:48:15.845706 27213 rpcz_store.cc:269] 0807 01:48:13.090858 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:13.838722 (+747864us) service_pool.cc:224] Handling call
0807 01:48:15.729974 (+1891252us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b54001de054. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:15.730137 (+ 163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":641845632}
W20260807 01:48:15.734290 27108 rpcz_store.cc:269] 0807 01:48:13.883152 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.142500 (+259348us) service_pool.cc:224] Handling call
0807 01:48:15.676081 (+1533581us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b54000401d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:15.676262 (+ 181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":335433088}
I20260807 01:48:15.735553 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:15.845446 27326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59490 (request call id 2678) took 2829 ms. Trace:
I20260807 01:48:15.742903 27338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33976 (request call id 2665) took 2709 ms. Trace:
I20260807 01:48:15.705911 32154 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:15.695376 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:15.717039 31954 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:15.754781 27101 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 1796) took 1872 ms. Trace:
I20260807 01:48:15.873667 27101 rpcz_store.cc:276] 0807 01:48:13.882602 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:15.636576 (+1753974us) service_pool.cc:224] Handling call
0807 01:48:15.754709 (+118133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:15.875502 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:48:15.760000 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 110455us
I20260807 01:48:15.762791 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:15.877080 27105 raft_consensus.cc:2410] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260807 01:48:15.877367 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:15.767226 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:15.878252 27333 raft_consensus.cc:2468] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
I20260807 01:48:15.878576 27287 leader_election.cc:400] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 6. Current term is 7.
I20260807 01:48:15.733276 32179 leader_election.cc:290] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:15.830857 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 188447us
I20260807 01:48:15.834426 27218 raft_consensus.cc:2393] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 7: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:15.736886 31632 raft_consensus.cc:493] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:15.746030 32108 leader_election.cc:290] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:15.848861 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:15.887146 27108 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:36914
I20260807 01:48:15.887880 27108 raft_consensus.cc:606] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Received request to transfer leadership to TS 3478729505b046c2a0051c2d0130873f
I20260807 01:48:15.888212 27108 raft_consensus.cc:614] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 01:48:15.840864 28707 rpcz_store.cc:276] 0807 01:48:11.961219 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:11.961383 (+ 164us) service_pool.cc:224] Handling call
0807 01:48:11.961789 (+ 406us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:11.983098 (+ 21309us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:11.983148 (+ 50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:12.037253 (+ 54105us) log.cc:844] Serialized 26 byte log entry
0807 01:48:13.845641 (+1808388us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b1800358900. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d5b233f31 00007f7d5b2406d1 00007f7d522fa7b7 00007f7d5b1a2fc9 00007f7d52316fdb 00007f7d523152db 00007f7d52317542 00007f7d54fb73fb 00007f7d54fbcf13 00007f7d54fbd2c1 00007f7d553d9def
0807 01:48:14.654919 (+809278us) raft_consensus.cc:1645] Marking committed up to 10
0807 01:48:14.654962 (+ 43us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.657213 (+ 2251us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:15.685066 (+1027853us) raft_consensus.cc:1685] finished
0807 01:48:15.685086 (+ 20us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:15.713834 (+ 28748us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50912,"raft.queue_time_us":21131,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":48486,"spinlock_wait_cycles":25126784,"thread_start_us":809052,"threads_started":1,"wal-append.queue_time_us":3143908}
I20260807 01:48:15.888901 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:15.847491 27338 rpcz_store.cc:276] 0807 01:48:13.033475 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.526563 (+1493088us) service_pool.cc:224] Handling call
0807 01:48:14.531569 (+ 5006us) raft_consensus.cc:1471] Updating replica for 2 ops
0807 01:48:14.598749 (+ 67180us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:14.598785 (+ 36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0807 01:48:14.621490 (+ 22705us) log.cc:844] Serialized 52 byte log entry
0807 01:48:14.871113 (+249623us) spinlock_profiling.cc:243] Waited 1.88 ms on lock 0x7b18003d98a0. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d5b233f31 00007f7d5b2406d1 00007f7d522fa7b7 00007f7d5b1a2fc9 00007f7d52316fdb 00007f7d523152db 00007f7d52317542 00007f7d54fb73fb 00007f7d54fbcf13 00007f7d54fbd2c1 00007f7d553d9def
0807 01:48:15.469625 (+598512us) raft_consensus.cc:1645] Marking committed up to 3
0807 01:48:15.597974 (+128349us) log.cc:844] Serialized 14 byte log entry
0807 01:48:15.598196 (+ 222us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:15.600940 (+ 2744us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:15.717123 (+116183us) raft_consensus.cc:1685] finished
0807 01:48:15.742591 (+ 25468us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:15.742810 (+ 219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37155,"prepare.queue_time_us":188894,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":128674,"raft.run_cpu_time_us":506,"raft.run_wall_time_us":48746,"spinlock_wait_cycles":4333440,"thread_start_us":678356,"threads_started":2,"wal-append.queue_time_us":995701}
I20260807 01:48:15.762120 32264 raft_consensus.cc:493] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:15.847067 27326 rpcz_store.cc:276] 0807 01:48:13.015864 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.301452 (+1285588us) service_pool.cc:224] Handling call
0807 01:48:14.301805 (+ 353us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:14.322235 (+ 20430us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:14.322272 (+ 37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:14.322812 (+ 540us) log.cc:844] Serialized 26 byte log entry
0807 01:48:14.863457 (+540645us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:48:14.863502 (+ 45us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.863572 (+ 70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:15.845210 (+981638us) raft_consensus.cc:1685] finished
0807 01:48:15.845234 (+ 24us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:15.845386 (+ 152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":21456,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":17472,"thread_start_us":445089,"threads_started":1,"wal-append.queue_time_us":919898}
I20260807 01:48:15.752383 27110 raft_consensus.cc:2468] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:48:15.859587 32254 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:15.855949 31896 raft_consensus.cc:3060] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:15.894843 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:15.856578 27328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33976 (request call id 2669) took 2822 ms. Trace:
I20260807 01:48:15.857172 31652 raft_consensus.cc:515] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:15.833462 32135 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:15.859380 27226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57662 (request call id 2812) took 2788 ms. Trace:
I20260807 01:48:15.746706 31453 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:15.754709 32071 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:15.853540 32166 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:15.863729 32186 raft_consensus.cc:3060] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:15.864672 32223 raft_consensus.cc:3055] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:48:15.856195 27103 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 695) took 2765 ms. Trace:
I20260807 01:48:15.932904 27103 rpcz_store.cc:276] 0807 01:48:13.091112 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:13.891754 (+800642us) service_pool.cc:224] Handling call
0807 01:48:13.892288 (+ 534us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:13.909488 (+ 17200us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:13.909529 (+ 41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:13.910319 (+ 790us) log.cc:844] Serialized 26 byte log entry
0807 01:48:14.459815 (+549496us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b180006cb80. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d5b233f31 00007f7d5b2406d1 00007f7d522fa7b7 00007f7d5b1a2fc9 00007f7d52316fdb 00007f7d523152db 00007f7d52317542 00007f7d54fb73fb 00007f7d54fbcf13 00007f7d54fbd2c1 00007f7d553d9def
0807 01:48:14.924999 (+465184us) raft_consensus.cc:1645] Marking committed up to 8
0807 01:48:15.001356 (+ 76357us) log.cc:844] Serialized 14 byte log entry
0807 01:48:15.001613 (+ 257us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:15.001697 (+ 84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:15.855855 (+854158us) raft_consensus.cc:1685] finished
0807 01:48:15.855963 (+ 108us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:15.856124 (+ 161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16621,"prepare.queue_time_us":65940,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":9479,"raft.queue_time_us":18125,"raft.run_cpu_time_us":433,"raft.run_wall_time_us":17120,"spinlock_wait_cycles":6306560,"thread_start_us":464960,"threads_started":1,"wal-append.queue_time_us":1479089}
I20260807 01:48:15.865629 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:15.935806 27230 raft_consensus.cc:2410] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260807 01:48:15.866519 32260 log.cc:927] Time spent T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.210s user 0.001s sys 0.000s
I20260807 01:48:15.865929 31953 consensus_queue.cc:1059] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:48:15.866086 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:15.857975 32140 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:15.868813 27100 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 3099) took 2782 ms. Trace:
I20260807 01:48:15.942836 27100 rpcz_store.cc:276] 0807 01:48:13.086514 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:13.718907 (+632393us) service_pool.cc:224] Handling call
0807 01:48:13.719199 (+ 292us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:15.262446 (+1543247us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:15.868555 (+606109us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54001711d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:15.868746 (+ 191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2807892224}
I20260807 01:48:15.837693 32262 raft_consensus.cc:2749] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260807 01:48:15.871531 32043 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:15.872340 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:15.736123 32265 consensus_queue.cc:1059] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.125s
I20260807 01:48:15.950708 27102 raft_consensus.cc:2468] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:48:15.950876 32265 consensus_queue.cc:1243] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer fcf4e45defb94276bcfa68392711dade log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260807 01:48:15.951665 32265 raft_consensus.cc:3055] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 9 LEADER]: Stepping down as leader of term 9
I20260807 01:48:15.952051 32265 raft_consensus.cc:740] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:15.953032 32265 consensus_queue.cc:260] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:15.954757 32265 raft_consensus.cc:3060] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Advancing to term 10
I20260807 01:48:15.955329 31652 leader_election.cc:290] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:15.892252 31632 raft_consensus.cc:515] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:15.957911 31632 leader_election.cc:290] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:15.940835 31117 consensus_meta.cc:287] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 0.980s user 0.012s sys 0.000s
I20260807 01:48:15.958745 31117 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:15.961028 31117 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:15.951293 27102 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41080 (request call id 3111) took 1539 ms. Trace:
I20260807 01:48:15.962590 27102 rpcz_store.cc:276] 0807 01:48:14.411593 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:15.871998 (+1460405us) service_pool.cc:224] Handling call
0807 01:48:15.951227 (+ 79229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:15.964003 27172 leader_election.cc:304] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:15.970925 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:15.971824 27230 raft_consensus.cc:2393] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 5: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:15.953011 31453 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:15.935415 32249 raft_consensus.cc:697] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 5 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:15.973038 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:15.973351 32179 consensus_queue.cc:1059] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:48:15.877521 27105 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 1801) took 1758 ms. Trace:
I20260807 01:48:15.974529 32179 consensus_queue.cc:1059] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 01:48:15.975871 32163 raft_consensus.cc:3055] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 8 LEADER]: Stepping down as leader of term 8
W20260807 01:48:15.706765 28661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:15.976316 32163 raft_consensus.cc:740] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:15.973678 32249 consensus_queue.cc:237] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:15.977308 32163 consensus_queue.cc:260] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:15.785554 27067 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.4.4:46225 timed out after 1.743s (SENT) [suppressed 27 similar messages]
I20260807 01:48:15.977979 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:15.978596 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 193109us
I20260807 01:48:15.879266 27287 leader_election.cc:403] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260807 01:48:15.980618 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:15.766633 31943 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:15.981568 28708 raft_consensus.cc:2410] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260807 01:48:15.983480 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:15.880788 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:15.984387 28710 raft_consensus.cc:1240] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 9 LEADER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 8. Current term is 9. Ops: []
I20260807 01:48:15.881296 27214 raft_consensus.cc:2393] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 10: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:15.985121 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:15.854473 27099 raft_consensus.cc:2468] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:48:15.890456 27110 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 1803) took 1770 ms. Trace:
I20260807 01:48:15.853125 31948 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:15.985625 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:15.845343 31771 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:15.884613 27213 raft_consensus.cc:1240] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 13 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 12. Current term is 13. Ops: []
I20260807 01:48:15.905545 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:15.895570 27328 rpcz_store.cc:276] 0807 01:48:13.034499 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.531374 (+1496875us) service_pool.cc:224] Handling call
0807 01:48:14.560424 (+ 29050us) raft_consensus.cc:1471] Updating replica for 2 ops
0807 01:48:14.574493 (+ 14069us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:14.574528 (+ 35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0807 01:48:14.611941 (+ 37413us) log.cc:844] Serialized 52 byte log entry
0807 01:48:15.335192 (+723251us) raft_consensus.cc:1645] Marking committed up to 5
0807 01:48:15.335231 (+ 39us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:15.335304 (+ 73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:15.856345 (+521041us) raft_consensus.cc:1685] finished
0807 01:48:15.856366 (+ 21us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:15.856520 (+ 154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13287,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":12182,"thread_start_us":664948,"threads_started":1,"wal-append.queue_time_us":805581}
I20260807 01:48:15.985693 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:15.987907 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:15.918365 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:15.989027 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:15.989544 31943 raft_consensus.cc:493] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:15.989065 27101 raft_consensus.cc:2393] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 7: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:15.974342 27105 rpcz_store.cc:276] 0807 01:48:14.119169 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:15.762498 (+1643329us) service_pool.cc:224] Handling call
0807 01:48:15.877459 (+114961us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:15.991604 27101 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 703) took 2874 ms. Trace:
W20260807 01:48:15.885283 32216 log.cc:927] Time spent T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.051s user 0.000s sys 0.000s
I20260807 01:48:15.992046 27067 leader_election.cc:304] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:15.921190 27226 rpcz_store.cc:276] 0807 01:48:13.070870 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.167809 (+1096939us) service_pool.cc:224] Handling call
0807 01:48:14.168200 (+ 391us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:14.777026 (+608826us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:14.777063 (+ 37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:14.806216 (+ 29153us) log.cc:844] Serialized 26 byte log entry
0807 01:48:15.279013 (+472797us) raft_consensus.cc:1645] Marking committed up to 3
0807 01:48:15.279059 (+ 46us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:15.279143 (+ 84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:15.859162 (+580019us) raft_consensus.cc:1685] finished
0807 01:48:15.859195 (+ 33us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:15.859319 (+ 124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":250881,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":17859,"thread_start_us":1069152,"threads_started":2,"wal-append.queue_time_us":651448}
I20260807 01:48:15.993371 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:15.986851 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:15.929690 27333 pending_rounds.cc:85] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade: Aborting all ops after (but not including) 7
I20260807 01:48:15.937898 27108 raft_consensus.cc:1275] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260807 01:48:15.986304 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:15.941915 27103 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 698) took 2839 ms. Trace:
I20260807 01:48:15.946092 27227 raft_consensus.cc:2393] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 9: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:15.890918 32264 raft_consensus.cc:515] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:15.984704 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:15.949487 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:15.987855 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:15.985800 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:15.994122 27104 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 694) took 2903 ms. Trace:
I20260807 01:48:15.974802 28711 raft_consensus.cc:2468] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
W20260807 01:48:15.988566 24592 auto_leader_rebalancer.cc:467] leader transfer for tablet 565d0a4463ac45e4bad1886cc0095ff7 (from 3262a9d183914b699227fdba0ca5bed3 to 3478729505b046c2a0051c2d0130873f) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 01:48:15.995047 27108 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 697) took 2903 ms. Trace:
I20260807 01:48:15.987236 27099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 1806) took 1856 ms. Trace:
I20260807 01:48:15.890079 28713 raft_consensus.cc:1240] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 7. Current term is 8. Ops: []
I20260807 01:48:15.857957 32151 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:15.990064 31943 raft_consensus.cc:515] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:16.004477 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:16.005312 31943 leader_election.cc:290] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:15.917723 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:16.006572 27100 raft_consensus.cc:1275] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260807 01:48:15.997502 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 119104us
I20260807 01:48:15.988637 28709 raft_consensus.cc:2393] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 8: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:15.997068 27212 raft_consensus.cc:2368] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6.
I20260807 01:48:15.994463 27327 raft_consensus.cc:2393] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 4: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:15.984444 27109 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41080 (request call id 3114) took 1572 ms. Trace:
I20260807 01:48:16.002698 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:16.003427 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:15.999590 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:16.009698 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:15.986482 27110 rpcz_store.cc:276] 0807 01:48:14.119685 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:15.632754 (+1513069us) service_pool.cc:224] Handling call
0807 01:48:15.890388 (+257634us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:15.997433 27333 pending_rounds.cc:107] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260807 01:48:16.001219 27338 raft_consensus.cc:1275] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 01:48:15.994570 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:16.001152 27108 rpcz_store.cc:276] 0807 01:48:13.091839 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:15.937170 (+2845331us) service_pool.cc:224] Handling call
0807 01:48:15.937541 (+ 371us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:15.994810 (+ 57269us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:15.994994 (+ 184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:16.002732 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:15.996860 27227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57654 (request call id 1942) took 1195 ms. Trace:
I20260807 01:48:16.012243 27333 raft_consensus.cc:2889] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 01:48:16.001844 27099 rpcz_store.cc:276] 0807 01:48:14.131067 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:15.694676 (+1563609us) service_pool.cc:224] Handling call
0807 01:48:15.987173 (+292497us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:16.013695 27227 rpcz_store.cc:276] 0807 01:48:14.801654 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.801792 (+ 138us) service_pool.cc:224] Handling call
0807 01:48:15.996683 (+1194891us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54001df954. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:15.996803 (+ 120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2603632512}
I20260807 01:48:15.995177 27103 rpcz_store.cc:276] 0807 01:48:13.103127 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:15.941282 (+2838155us) service_pool.cc:224] Handling call
0807 01:48:15.941543 (+ 261us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:15.941693 (+ 150us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:15.941707 (+ 14us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:15.941739 (+ 32us) raft_consensus.cc:1645] Marking committed up to 7
0807 01:48:15.941764 (+ 25us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:15.941799 (+ 35us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:15.941870 (+ 71us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:16.011878 27226 raft_consensus.cc:2376] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7. Re-sending same reply.
I20260807 01:48:16.016908 27226 raft_consensus.cc:1275] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260807 01:48:16.010659 27109 rpcz_store.cc:276] 0807 01:48:14.412350 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:15.870089 (+1457739us) service_pool.cc:224] Handling call
0807 01:48:15.984382 (+114293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:16.020117 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:15.991971 27101 rpcz_store.cc:276] 0807 01:48:13.117390 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:15.990756 (+2873366us) service_pool.cc:224] Handling call
0807 01:48:15.991121 (+ 365us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:15.991301 (+ 180us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:15.991324 (+ 23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:15.991365 (+ 41us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:48:15.991403 (+ 38us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:15.991454 (+ 51us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:15.991546 (+ 92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10496}
I20260807 01:48:15.986980 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:16.022218 28710 raft_consensus.cc:2393] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 4: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:15.998883 27213 raft_consensus.cc:2376] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6. Re-sending same reply.
I20260807 01:48:16.012161 27110 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 705) took 2765 ms. Trace:
I20260807 01:48:16.023355 27110 rpcz_store.cc:276] 0807 01:48:13.247040 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:16.011140 (+2764100us) service_pool.cc:224] Handling call
0807 01:48:16.011514 (+ 374us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:16.011820 (+ 306us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:16.011849 (+ 29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:16.011896 (+ 47us) raft_consensus.cc:1645] Marking committed up to 7
0807 01:48:16.011934 (+ 38us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:16.011992 (+ 58us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:16.012103 (+ 111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:16.009588 27289 leader_election.cc:304] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:16.007732 27287 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:16.026731 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:15.999464 27104 rpcz_store.cc:276] 0807 01:48:13.090856 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:13.875789 (+784933us) service_pool.cc:224] Handling call
0807 01:48:13.876176 (+ 387us) raft_consensus.cc:1471] Updating replica for 2 ops
0807 01:48:13.882180 (+ 6004us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:13.882212 (+ 32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0807 01:48:13.882858 (+ 646us) log.cc:844] Serialized 52 byte log entry
0807 01:48:14.787653 (+904795us) raft_consensus.cc:1645] Marking committed up to 7
0807 01:48:14.816997 (+ 29344us) log.cc:844] Serialized 14 byte log entry
0807 01:48:14.831206 (+ 14209us) log.cc:844] Serialized 14 byte log entry
0807 01:48:14.831489 (+ 283us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:14.831561 (+ 72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:15.674775 (+843214us) spinlock_profiling.cc:243] Waited 5.46 ms on lock 0x7b1800168b80. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d5b233f31 00007f7d5b2406d1 00007f7d52317d33 00007f7d5231a121 00007f7d5231a0d9 00007f7d5231a069 00007f7d5231a031 00007f7d5231932d 00000000003aa554 00000000003aa399 00007f7d522fb1ea
0807 01:48:15.993898 (+319123us) raft_consensus.cc:1685] finished
0807 01:48:15.993919 (+ 21us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:15.994054 (+ 135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19224,"prepare.queue_time_us":68764,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":2421,"raft.queue_time_us":27810,"raft.run_cpu_time_us":826,"raft.run_wall_time_us":27914,"spinlock_wait_cycles":12559744,"thread_start_us":864858,"threads_started":1,"wal-append.queue_time_us":1111330}
I20260807 01:48:16.027596 27214 raft_consensus.cc:2393] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 7: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:16.028625 27104 raft_consensus.cc:1275] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 01:48:16.029304 27104 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 3113) took 1617 ms. Trace:
I20260807 01:48:16.029630 27104 rpcz_store.cc:276] 0807 01:48:14.412114 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:16.028005 (+1615891us) service_pool.cc:224] Handling call
0807 01:48:16.028314 (+ 309us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:16.029022 (+ 708us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:16.029240 (+ 218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:15.997916 27064 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:16.010612 28715 raft_consensus.cc:2468] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
W20260807 01:48:16.027436 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 63539.3us
I20260807 01:48:16.037905 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:16.040285 27108 raft_consensus.cc:1240] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 9. Current term is 10. Ops: []
I20260807 01:48:16.041143 27108 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 707) took 2222 ms. Trace:
I20260807 01:48:16.041507 27108 rpcz_store.cc:276] 0807 01:48:13.836036 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:16.039668 (+2203632us) service_pool.cc:224] Handling call
0807 01:48:16.040068 (+ 400us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:16.040847 (+ 779us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:16.041074 (+ 227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:16.044691 27104 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 3115) took 1632 ms. Trace:
I20260807 01:48:16.045570 27104 rpcz_store.cc:276] 0807 01:48:14.412582 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:16.030553 (+1617971us) service_pool.cc:224] Handling call
0807 01:48:16.031859 (+ 1306us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:16.044284 (+ 12425us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:16.044317 (+ 33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:16.044370 (+ 53us) raft_consensus.cc:1645] Marking committed up to 4
0807 01:48:16.044413 (+ 43us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:16.044485 (+ 72us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:16.044609 (+ 124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11998}
W20260807 01:48:16.043021 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 45171.9us
I20260807 01:48:16.050967 27064 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:16.044229 27289 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:15.932655 32223 raft_consensus.cc:740] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:16.004730 32151 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:15.980568 32174 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:15.994711 32271 log.cc:927] Time spent T 81439f7f101341049bd086b353ca3495 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.153s user 0.000s sys 0.000s
I20260807 01:48:16.071147 32223 consensus_queue.cc:260] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:16.054958 32130 raft_consensus.cc:2804] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:48:16.073158 32130 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:16.073545 32130 raft_consensus.cc:3060] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Advancing to term 8
W20260807 01:48:15.995647 32199 log.cc:927] Time spent T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 1.256s user 0.006s sys 0.006s
I20260807 01:48:16.005934 31943 raft_consensus.cc:2749] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 6. Current term is 7.
I20260807 01:48:16.083508 32071 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:15.996474 32166 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:16.086272 32223 raft_consensus.cc:3060] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:16.066808 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:15.983851 32141 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:16.088686 28708 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 8.
I20260807 01:48:15.976379 31960 raft_consensus.cc:3055] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 8 LEADER]: Stepping down as leader of term 8
W20260807 01:48:16.089916 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 82277.4us
I20260807 01:48:16.087790 27099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1802) took 1968 ms. Trace:
I20260807 01:48:16.089931 27101 raft_consensus.cc:1240] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 3. Current term is 4. Ops: []
I20260807 01:48:16.001940 32163 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Advancing to term 9
W20260807 01:48:16.089162 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 45064.7us
I20260807 01:48:16.065768 27110 raft_consensus.cc:1275] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260807 01:48:16.091230 27287 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:16.091996 27101 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 3110) took 1681 ms. Trace:
W20260807 01:48:16.102314 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 446433us
I20260807 01:48:16.103587 27101 rpcz_store.cc:276] 0807 01:48:14.411281 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:16.021368 (+1610087us) service_pool.cc:224] Handling call
0807 01:48:16.089765 (+ 68397us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:16.091749 (+ 1984us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:16.091938 (+ 189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:16.104337 28660 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:16.100827 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:16.101857 27108 raft_consensus.cc:1240] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 4. Current term is 5. Ops: []
I20260807 01:48:16.025887 31771 raft_consensus.cc:2749] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 01:48:16.103130 27110 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 3112) took 1691 ms. Trace:
I20260807 01:48:16.105962 27110 rpcz_store.cc:276] 0807 01:48:14.411865 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:16.024418 (+1612553us) service_pool.cc:224] Handling call
0807 01:48:16.065346 (+ 40928us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:16.102855 (+ 37509us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:16.103066 (+ 211us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 01:48:16.094820 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 60723.1us
I20260807 01:48:16.091194 27099 rpcz_store.cc:276] 0807 01:48:14.119440 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:16.042272 (+1922832us) service_pool.cc:224] Handling call
0807 01:48:16.087229 (+ 44957us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:16.087432 (+ 203us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:16.087460 (+ 28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:16.087503 (+ 43us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:48:16.087539 (+ 36us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:16.087591 (+ 52us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:16.087723 (+ 132us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 01:48:16.107781 27170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:16.110559 27108 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 712) took 1370 ms. Trace:
I20260807 01:48:16.113523 27108 rpcz_store.cc:276] 0807 01:48:14.739641 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:16.044996 (+1305355us) service_pool.cc:224] Handling call
0807 01:48:16.101660 (+ 56664us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:16.110009 (+ 8349us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:16.110429 (+ 420us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:15.986227 31948 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:16.061226 31632 raft_consensus.cc:2804] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:48:16.133689 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:16.135247 32110 raft_consensus.cc:2749] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 01:48:16.137465 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:16.139043 27213 raft_consensus.cc:2468] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
I20260807 01:48:16.133361 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:16.141870 28660 leader_election.cc:304] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters: 3478729505b046c2a0051c2d0130873f
W20260807 01:48:16.142916 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 51772us
I20260807 01:48:16.094043 32146 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:16.147761 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.155959 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.156584 27330 raft_consensus.cc:2393] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 6: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:16.145597 28660 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:16.145781 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.146373 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.146574 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.147156 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:16.179881 27325 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 12 FOLLOWER]: Advancing to term 13
I20260807 01:48:16.180115 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:16.149207 32274 raft_consensus.cc:2804] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:48:16.129670 32279 raft_consensus.cc:2804] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Leader election won for term 10
I20260807 01:48:16.099264 32139 raft_consensus.cc:2749] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260807 01:48:16.151294 31632 raft_consensus.cc:697] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [term 7 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:16.152805 27101 raft_consensus.cc:1240] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 7. Current term is 8. Ops: []
I20260807 01:48:16.201928 27101 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 3124) took 1095 ms. Trace:
I20260807 01:48:16.202366 27101 rpcz_store.cc:276] 0807 01:48:15.106999 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:16.131859 (+1024860us) service_pool.cc:224] Handling call
0807 01:48:16.152622 (+ 20763us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:16.201600 (+ 48978us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:16.201855 (+ 255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:16.144263 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:16.202970 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:16.203765 27212 raft_consensus.cc:2368] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for earlier term 4. Current term is 5.
I20260807 01:48:16.180138 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:16.142005 27214 raft_consensus.cc:2376] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 8. Re-sending same reply.
W20260807 01:48:16.206413 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 153484us
I20260807 01:48:16.181102 27327 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:16.208663 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.093180 31960 raft_consensus.cc:740] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:16.186074 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:16.188689 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:48:16.199450 28662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:16.212263 27337 raft_consensus.cc:2393] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 10: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
W20260807 01:48:16.212369 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 66875.1us
I20260807 01:48:16.210628 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:16.213187 28660 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:16.215706 27212 raft_consensus.cc:1275] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 01:48:16.217522 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:16.218326 27331 raft_consensus.cc:3060] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:16.213933 27325 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 13.
I20260807 01:48:16.223894 27337 raft_consensus.cc:1240] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6. Ops: [3.3-5.5]
I20260807 01:48:16.225170 27328 raft_consensus.cc:1275] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 01:48:16.221343 27067 leader_election.cc:304] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:16.185087 32144 leader_election.cc:290] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:16.230350 27326 raft_consensus.cc:1275] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260807 01:48:16.234627 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.190871 32110 raft_consensus.cc:2804] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 01:48:16.244092 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.246372 27336 raft_consensus.cc:2393] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 7: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:16.240839 27327 raft_consensus.cc:2468] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 9.
I20260807 01:48:16.242219 27331 raft_consensus.cc:2468] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:48:16.234362 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.247468 27337 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 9.
I20260807 01:48:16.142846 31117 consensus_queue.cc:1059] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260807 01:48:16.153443 31931 consensus_queue.cc:1059] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260807 01:48:16.249781 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:16.244880 27328 raft_consensus.cc:2468] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:16.260145 27337 raft_consensus.cc:2393] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 5: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:16.260393 31117 raft_consensus.cc:3055] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 01:48:16.260941 31117 raft_consensus.cc:740] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
W20260807 01:48:16.261976 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 130256us
W20260807 01:48:16.262259 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 49128us
I20260807 01:48:16.265334 28660 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:16.266569 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.266598 28660 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:16.267458 27331 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8.
I20260807 01:48:16.271078 27109 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 3108) took 1869 ms. Trace:
I20260807 01:48:16.214905 32279 raft_consensus.cc:697] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [term 10 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:16.271868 27109 rpcz_store.cc:276] 0807 01:48:14.401893 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:16.020297 (+1618404us) service_pool.cc:224] Handling call
0807 01:48:16.068051 (+ 47754us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:16.068470 (+ 419us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:16.068496 (+ 26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:16.139242 (+ 70746us) log.cc:844] Serialized 540 byte log entry
0807 01:48:16.260426 (+121184us) raft_consensus.cc:1645] Marking committed up to 4
0807 01:48:16.260468 (+ 42us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:16.260553 (+ 85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:16.270750 (+ 10197us) raft_consensus.cc:1685] finished
0807 01:48:16.270778 (+ 28us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:16.270994 (+ 216us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":120839,"threads_started":1,"wal-append.queue_time_us":120785}
I20260807 01:48:16.224313 32274 raft_consensus.cc:697] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 7 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:16.274724 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:16.275501 27337 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:16.275588 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.233893 31509 raft_consensus.cc:2804] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:16.227896 32153 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:16.278040 27101 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b"
dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36914
W20260807 01:48:16.279233 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 190810us
I20260807 01:48:16.227242 32286 raft_consensus.cc:2804] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 01:48:16.284710 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:16.283305 27101 raft_consensus.cc:606] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:48:16.285533 27101 raft_consensus.cc:614] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 01:48:16.245275 32106 raft_consensus.cc:2804] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 01:48:16.281667 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.286723 27102 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 700) took 3173 ms. Trace:
I20260807 01:48:16.287564 27102 rpcz_store.cc:276] 0807 01:48:13.113773 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:15.963365 (+2849592us) service_pool.cc:224] Handling call
0807 01:48:15.963708 (+ 343us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:15.963933 (+ 225us) raft_consensus.cc:1534] Early marking committed up to index 9
0807 01:48:15.964254 (+ 321us) log.cc:844] Serialized 14 byte log entry
0807 01:48:16.284754 (+320500us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:16.284864 (+ 110us) raft_consensus.cc:1645] Marking committed up to 9
0807 01:48:16.284940 (+ 76us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:16.285148 (+ 208us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:16.286608 (+ 1460us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11979,"prepare.queue_time_us":596,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"thread_start_us":307982,"threads_started":1}
I20260807 01:48:16.289717 27064 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:16.291791 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:16.292426 27170 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:48:16.292520 27331 raft_consensus.cc:2376] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 13. Re-sending same reply.
I20260807 01:48:16.259016 32264 leader_election.cc:290] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:16.270979 31954 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:16.271183 32110 raft_consensus.cc:697] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
W20260807 01:48:16.308194 27287 connection.cc:256] RPC call timeout handler was delayed by -1.5215045940130949s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1955s for a response
I20260807 01:48:16.288297 31931 consensus_queue.cc:1243] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer 57bb4985157447f0a3ab680fa0026f12 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260807 01:48:16.283097 31509 raft_consensus.cc:493] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 01:48:16.313282 27287 connection.cc:256] RPC call timeout handler was delayed by -1.4990045940503478s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.198s for a response
I20260807 01:48:16.288595 32286 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:16.243108 32304 consensus_queue.cc:1059] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260807 01:48:16.288491 32039 raft_consensus.cc:2804] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:48:16.302364 32106 raft_consensus.cc:697] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 8 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:16.297206 32153 raft_consensus.cc:2804] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:16.325196 27337 raft_consensus.cc:2468] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 9.
I20260807 01:48:16.325731 27287 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
W20260807 01:48:16.327502 24592 auto_leader_rebalancer.cc:467] leader transfer for tablet 9db18f8163fc4d79aceac26f0e762e2b (from 3262a9d183914b699227fdba0ca5bed3 to fcf4e45defb94276bcfa68392711dade) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 01:48:16.303625 32071 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:16.329406 27064 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:16.328797 27287 leader_election.cc:304] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:16.335382 27064 leader_election.cc:400] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6.
I20260807 01:48:16.335745 27064 leader_election.cc:403] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260807 01:48:16.337159 27064 leader_election.cc:304] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:16.274073 31117 consensus_queue.cc:260] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:16.324738 31931 consensus_queue.cc:1059] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260807 01:48:16.296022 31960 consensus_queue.cc:260] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:16.341593 31117 raft_consensus.cc:3060] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:16.322510 32286 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:16.328276 32254 raft_consensus.cc:2749] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:16.339843 27287 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:16.322438 31509 raft_consensus.cc:3060] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:16.351274 32180 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:16.351822 32274 consensus_queue.cc:237] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:16.342759 32281 consensus_queue.cc:1175] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "fcf4e45defb94276bcfa68392711dade" responder_term: 6 status { last_received { term: 6 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 6 index: 7 } }
I20260807 01:48:16.339694 32229 raft_consensus.cc:2764] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260807 01:48:16.324919 31954 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:16.330013 32273 raft_consensus.cc:3055] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 01:48:16.326536 32153 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:16.295481 31632 consensus_queue.cc:237] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:16.381007 27287 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:16.379621 31931 raft_consensus.cc:3060] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:16.343099 32279 consensus_queue.cc:237] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:16.329931 32264 consensus_queue.cc:1059] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.053s
I20260807 01:48:16.340651 32174 raft_consensus.cc:2749] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 01:48:16.382444 31948 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:16.384469 32249 raft_consensus.cc:2749] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:48:16.393599 32180 raft_consensus.cc:3055] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 12 LEADER]: Stepping down as leader of term 12
I20260807 01:48:16.341876 32186 consensus_queue.cc:1048] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260807 01:48:16.425895 28708 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
mode: GRACEFUL
new_leader_uuid: "3478729505b046c2a0051c2d0130873f"
from {username='slave'} at 127.0.0.1:34614
I20260807 01:48:16.437155 28708 raft_consensus.cc:606] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Received request to transfer leadership to TS 3478729505b046c2a0051c2d0130873f
I20260807 01:48:16.427829 27287 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:16.422205 32153 raft_consensus.cc:3060] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:16.439361 27287 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
W20260807 01:48:16.433292 32287 log.cc:927] Time spent T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.149s user 0.002s sys 0.000s
W20260807 01:48:16.435042 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 97962.4us
I20260807 01:48:16.430371 32254 raft_consensus.cc:2804] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260807 01:48:16.434130 32193 log.cc:927] Time spent T 51cef821f0284d8e9369e98d38f80e73 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 1.834s user 0.003s sys 0.004s
I20260807 01:48:16.442824 27064 leader_election.cc:304] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:16.424393 32273 raft_consensus.cc:740] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
W20260807 01:48:16.424300 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 151632us
I20260807 01:48:16.445823 27287 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:16.399144 32110 consensus_queue.cc:237] T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:16.444679 31743 leader_election.cc:290] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:16.394965 32106 consensus_queue.cc:237] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:16.462392 28661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:16.462781 32291 log.cc:927] Time spent T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.130s user 0.000s sys 0.000s
W20260807 01:48:16.464974 32297 log.cc:927] Time spent T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.088s user 0.000s sys 0.000s
I20260807 01:48:16.469611 27104 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41080 (request call id 3118) took 1542 ms. Trace:
I20260807 01:48:16.434772 32180 raft_consensus.cc:740] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:16.470713 27064 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:16.423560 32299 raft_consensus.cc:2749] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:48:16.470033 27104 rpcz_store.cc:276] 0807 01:48:14.927101 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:16.046938 (+1119837us) service_pool.cc:224] Handling call
0807 01:48:16.047360 (+ 422us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:16.086082 (+ 38722us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:16.086112 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:16.086654 (+ 542us) log.cc:844] Serialized 26 byte log entry
0807 01:48:16.288445 (+201791us) raft_consensus.cc:1645] Marking committed up to 2
0807 01:48:16.288486 (+ 41us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:16.288553 (+ 67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:16.469375 (+180822us) raft_consensus.cc:1685] finished
0807 01:48:16.469397 (+ 22us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:16.469545 (+ 148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19144,"raft.queue_time_us":84391,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":11323,"thread_start_us":160531,"threads_started":1,"wal-append.queue_time_us":303037}
I20260807 01:48:16.472846 27327 raft_consensus.cc:1240] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 13 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 12. Current term is 13. Ops: []
I20260807 01:48:16.473197 27331 raft_consensus.cc:1240] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 8. Current term is 9. Ops: []
I20260807 01:48:16.473846 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.483978 27326 raft_consensus.cc:2393] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 5: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:16.474186 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.477694 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.484337 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.484920 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.485210 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:16.484366 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:16.485494 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
W20260807 01:48:16.443676 32284 log.cc:927] Time spent T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.185s user 0.001s sys 0.000s
I20260807 01:48:16.486003 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:16.479110 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:48:16.462188 32296 log.cc:927] Time spent T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.106s user 0.001s sys 0.000s
I20260807 01:48:16.486349 27327 raft_consensus.cc:2376] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7. Re-sending same reply.
I20260807 01:48:16.486081 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.486650 27104 raft_consensus.cc:2376] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 8. Re-sending same reply.
I20260807 01:48:16.487449 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.487874 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.488425 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:16.488636 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.489042 27328 raft_consensus.cc:2376] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7. Re-sending same reply.
I20260807 01:48:16.486542 27330 raft_consensus.cc:2468] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:48:16.488374 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.491148 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:16.491482 27325 raft_consensus.cc:2376] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7. Re-sending same reply.
I20260807 01:48:16.492014 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:16.492380 27331 raft_consensus.cc:2376] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6. Re-sending same reply.
I20260807 01:48:16.493443 27109 raft_consensus.cc:1240] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260807 01:48:16.490248 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:16.493537 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:16.494966 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:16.495405 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:16.495342 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:16.496039 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:16.494225 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:16.442472 32229 raft_consensus.cc:2749] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:48:16.494462 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:16.491587 27102 raft_consensus.cc:3055] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 8 LEADER]: Stepping down as leader of term 8
I20260807 01:48:16.498580 27102 raft_consensus.cc:740] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:16.496153 27104 raft_consensus.cc:3060] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:16.445713 31943 raft_consensus.cc:2749] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 01:48:16.500931 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.496688 27108 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8.
I20260807 01:48:16.445390 32254 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:16.499552 27102 consensus_queue.cc:260] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:16.469219 31896 consensus_meta.cc:287] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.551s user 0.009s sys 0.000s
I20260807 01:48:16.507241 32273 consensus_queue.cc:260] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:16.509135 27102 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:16.509486 27101 raft_consensus.cc:2393] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 6: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
W20260807 01:48:16.513142 28711 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48350 (request call id 932) took 1709 ms (client timeout 1702 ms). Trace:
I20260807 01:48:16.478737 32304 raft_consensus.cc:2804] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:48:16.514406 28708 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:34614
I20260807 01:48:16.479878 32264 raft_consensus.cc:2749] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260807 01:48:16.517994 28711 rpcz_store.cc:269] 0807 01:48:14.803224 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:14.803647 (+ 423us) service_pool.cc:224] Handling call
0807 01:48:16.512909 (+1709262us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400322dd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:16.513052 (+ 143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2666624384}
W20260807 01:48:16.512943 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 246398us
I20260807 01:48:16.478875 32174 consensus_queue.cc:1048] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260807 01:48:16.524358 28708 raft_consensus.cc:606] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
W20260807 01:48:16.519034 32298 log.cc:927] Time spent T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.187s user 0.001s sys 0.000s
I20260807 01:48:16.538090 28660 leader_election.cc:400] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for earlier term 4. Current term is 5.
I20260807 01:48:16.528045 27102 raft_consensus.cc:1275] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260807 01:48:16.516635 32254 raft_consensus.cc:3060] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:16.502223 32318 raft_consensus.cc:2804] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:48:16.557935 28660 leader_election.cc:403] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260807 01:48:16.558053 27102 pending_rounds.cc:85] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3: Aborting all ops after (but not including) 7
I20260807 01:48:16.570117 27102 pending_rounds.cc:107] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260807 01:48:16.570508 27102 raft_consensus.cc:2889] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 01:48:16.586764 24592 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 3
W20260807 01:48:16.588315 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 50397.6us
I20260807 01:48:16.589001 24592 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 01:48:16.589254 28660 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:16.539278 32304 raft_consensus.cc:697] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [term 7 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:16.558873 32273 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Advancing to term 8
W20260807 01:48:16.597393 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 99334.6us
I20260807 01:48:16.592877 32106 raft_consensus.cc:3060] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:16.613513 27104 raft_consensus.cc:2468] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 9.
I20260807 01:48:16.620862 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:16.620947 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:16.622373 28708 raft_consensus.cc:2393] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 5: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:16.622598 28711 raft_consensus.cc:2393] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 7: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
W20260807 01:48:16.614430 32130 consensus_meta.cc:287] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.540s user 0.009s sys 0.003s
I20260807 01:48:16.618216 31943 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:16.615533 32281 raft_consensus.cc:2804] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 01:48:16.657423 27103 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1799) took 2774 ms. Trace:
I20260807 01:48:16.542876 32180 consensus_queue.cc:260] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:16.627099 31943 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:16.657953 27103 rpcz_store.cc:276] 0807 01:48:13.883425 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:16.035796 (+2152371us) service_pool.cc:224] Handling call
0807 01:48:16.036362 (+ 566us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:16.045118 (+ 8756us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:16.045162 (+ 44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:16.052717 (+ 7555us) log.cc:844] Serialized 26 byte log entry
0807 01:48:16.233410 (+180693us) raft_consensus.cc:1645] Marking committed up to 10
0807 01:48:16.233453 (+ 43us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:16.233523 (+ 70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:16.657137 (+423614us) raft_consensus.cc:1685] finished
0807 01:48:16.657161 (+ 24us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:16.657349 (+ 188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65753,"raft.queue_time_us":16394,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":17095,"thread_start_us":180330,"threads_started":1,"wal-append.queue_time_us":242973}
I20260807 01:48:16.660715 32281 raft_consensus.cc:697] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 8 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
W20260807 01:48:16.664990 32323 log.cc:927] Time spent T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.072s user 0.000s sys 0.000s
I20260807 01:48:16.623370 32222 raft_consensus.cc:3055] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 7 LEADER]: Stepping down as leader of term 7
I20260807 01:48:16.677750 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.679625 27337 raft_consensus.cc:2468] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:48:16.650393 32130 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:16.679045 32222 raft_consensus.cc:740] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:16.621204 32265 consensus_queue.cc:1048] T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260807 01:48:16.694053 27063 leader_election.cc:304] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:16.695204 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:16.725240 27337 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
W20260807 01:48:16.710705 28662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:16.683794 32130 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:16.746201 31948 raft_consensus.cc:3055] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Stepping down as leader of term 6
I20260807 01:48:16.746757 31948 raft_consensus.cc:740] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:16.747689 31948 consensus_queue.cc:260] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:16.739253 27063 leader_election.cc:304] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:16.711790 32222 consensus_queue.cc:260] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:16.758622 32222 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:16.759047 31931 raft_consensus.cc:2749] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6.
I20260807 01:48:16.711974 31954 raft_consensus.cc:2749] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260807 01:48:16.713320 32223 consensus_meta.cc:287] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.619s user 0.008s sys 0.000s
I20260807 01:48:16.763839 27063 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:16.737979 31652 raft_consensus.cc:2749] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 01:48:16.789443 27100 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 699) took 3676 ms. Trace:
I20260807 01:48:16.790207 27100 rpcz_store.cc:276] 0807 01:48:13.113372 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:15.944255 (+2830883us) service_pool.cc:224] Handling call
0807 01:48:15.944593 (+ 338us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:16.006967 (+ 62374us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:16.788999 (+782032us) spinlock_profiling.cc:243] Waited 61.6 ms on lock 0x7b54003d68d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:16.789328 (+ 329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":141753216}
I20260807 01:48:16.701284 32254 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:16.780983 31931 raft_consensus.cc:2764] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260807 01:48:16.802362 32334 log.cc:927] Time spent T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.060s user 0.001s sys 0.000s
W20260807 01:48:16.786897 32163 consensus_meta.cc:287] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.658s user 0.009s sys 0.004s
I20260807 01:48:16.814615 31632 raft_consensus.cc:2749] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260807 01:48:16.705801 32153 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:16.828429 28708 raft_consensus.cc:1240] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 5. Current term is 6. Ops: []
I20260807 01:48:16.718892 32281 consensus_queue.cc:237] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:16.706422 32304 consensus_queue.cc:237] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:16.801919 31509 raft_consensus.cc:515] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:16.870081 32286 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:16.872697 32286 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:16.877242 32153 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499)
I20260807 01:48:16.878297 32224 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:16.878890 32254 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:16.944599 27064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:16.952049 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 157406us
W20260807 01:48:16.958076 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:16.960250 27105 raft_consensus.cc:1240] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260807 01:48:16.961521 27105 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 701) took 3847 ms. Trace:
I20260807 01:48:16.961925 27105 rpcz_store.cc:276] 0807 01:48:13.114254 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:15.990744 (+2876490us) service_pool.cc:224] Handling call
0807 01:48:16.959120 (+968376us) spinlock_profiling.cc:243] Waited 6.17 ms on lock 0x7b5800024c98. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e0521 00007f7d522e089e 000000000038d209 00007f7d5a39f940 00007f7d5a467ed4 00007f7d5a467f41 00007f7d5a429dd0 00007f7d5a414739 00007f7d5a4141e2 00007f7d53959d34
0807 01:48:16.959553 (+ 433us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:16.961043 (+ 1490us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:16.961391 (+ 348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14214784}
W20260807 01:48:16.967321 28661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:16.968580 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 364152us
W20260807 01:48:16.972759 27218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57662 (request call id 2825) took 1539 ms (client timeout 1578 ms). Trace:
W20260807 01:48:16.973474 27218 rpcz_store.cc:269] 0807 01:48:15.433202 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:15.450672 (+ 17470us) service_pool.cc:224] Handling call
0807 01:48:16.972010 (+1521338us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b54001f7cd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:16.972449 (+ 439us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":838212992}
W20260807 01:48:16.976575 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 684034us
I20260807 01:48:16.983922 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:16.984263 27172 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:16.985261 27227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57662 (request call id 2853) took 1008 ms. Trace:
I20260807 01:48:16.986202 27227 rpcz_store.cc:276] 0807 01:48:15.976480 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:16.018652 (+ 42172us) service_pool.cc:224] Handling call
0807 01:48:16.061694 (+ 43042us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:16.061981 (+ 287us) raft_consensus.cc:1534] Early marking committed up to index 3
0807 01:48:16.260827 (+198846us) log.cc:844] Serialized 14 byte log entry
0807 01:48:16.984363 (+723536us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:16.984531 (+ 168us) raft_consensus.cc:1645] Marking committed up to 3
0807 01:48:16.984695 (+ 164us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:16.984826 (+ 131us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:16.985053 (+ 227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":671566,"prepare.queue_time_us":73510,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":30573,"thread_start_us":212370,"threads_started":2,"wal-append.queue_time_us":116658}
I20260807 01:48:16.987996 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:16.992172 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:16.992813 28714 raft_consensus.cc:1240] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260807 01:48:16.994479 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:16.996024 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:16.996181 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:16.996281 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:16.997263 28702 raft_consensus.cc:2376] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6. Re-sending same reply.
I20260807 01:48:16.997979 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:16.998999 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:16.992136 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:17.000038 27102 raft_consensus.cc:2468] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:48:16.998909 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:17.000916 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:17.001448 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:17.001811 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:17.002844 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:17.009136 31203 raft_consensus.cc:2804] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:48:17.010366 31203 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:17.011075 31896 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:17.012428 31953 raft_consensus.cc:493] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:17.012275 31896 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:17.015005 31908 raft_consensus.cc:2749] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:16.997131 28711 raft_consensus.cc:1240] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 8 LEADER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 7. Current term is 8. Ops: []
I20260807 01:48:17.013463 31953 raft_consensus.cc:515] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:16.999471 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:17.006805 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:17.011075 31203 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:17.023615 32318 raft_consensus.cc:697] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
W20260807 01:48:17.024509 27064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:17.032014 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:17.027055 32274 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s user 0.037s sys 0.045s
W20260807 01:48:17.038058 32274 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.515s user 0.039s sys 0.045s
W20260807 01:48:17.041286 32328 log.cc:927] Time spent T 565d0a4463ac45e4bad1886cc0095ff7 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.268s user 0.004s sys 0.000s
W20260807 01:48:17.041544 32288 log.cc:927] Time spent T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.490s user 0.002s sys 0.000s
I20260807 01:48:17.039039 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:17.038391 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:17.048171 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:17.049672 27104 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 9.
I20260807 01:48:17.050292 28706 raft_consensus.cc:1275] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 01:48:17.053194 32286 consensus_queue.cc:1048] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260807 01:48:17.055337 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 65123.8us
W20260807 01:48:17.056002 32205 log.cc:927] Time spent T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 1.270s user 0.007s sys 0.000s
I20260807 01:48:17.056246 31953 leader_election.cc:290] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:17.056701 27105 raft_consensus.cc:2393] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 6: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:17.057204 28714 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
W20260807 01:48:17.061188 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 104757us
I20260807 01:48:17.066640 27102 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 9.
I20260807 01:48:17.058120 31896 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:17.076747 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:17.077335 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:17.084218 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:17.084647 32140 raft_consensus.cc:3055] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:48:17.085120 32140 raft_consensus.cc:740] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:17.085636 27330 raft_consensus.cc:2410] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
W20260807 01:48:17.088755 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 97780.2us
I20260807 01:48:17.089648 31509 leader_election.cc:290] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:17.077947 27287 leader_election.cc:304] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:17.091442 32186 raft_consensus.cc:2749] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 01:48:17.094815 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:17.095983 27227 raft_consensus.cc:2468] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
W20260807 01:48:17.097874 32349 log.cc:927] Time spent T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df: Append to log took a long time: real 0.184s user 0.000s sys 0.002s
I20260807 01:48:17.094846 27067 leader_election.cc:304] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:17.100203 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:17.101648 32106 raft_consensus.cc:2749] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for earlier term 4. Current term is 5.
I20260807 01:48:17.106602 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff03a52fbfd498ab6d04a579e96fd0e" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:17.107460 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:17.108007 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:17.109056 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:17.113662 27216 raft_consensus.cc:2468] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:48:17.113520 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:17.109731 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:17.120854 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:17.120795 32153 raft_consensus.cc:493] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:17.124548 27064 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:17.124514 28707 raft_consensus.cc:2468] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:48:17.125671 32151 raft_consensus.cc:2749] T 09799289505a4c188931e0f3ed0adfb3 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:17.126308 27330 raft_consensus.cc:2468] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
I20260807 01:48:17.124460 32153 raft_consensus.cc:515] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:17.126673 31509 raft_consensus.cc:2749] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 01:48:17.124614 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:17.133250 28706 raft_consensus.cc:3060] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:17.130084 27213 raft_consensus.cc:2468] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:48:17.139781 28714 raft_consensus.cc:1275] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 10 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260807 01:48:17.140196 32249 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:17.139876 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:17.141863 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:17.046757 32318 consensus_queue.cc:237] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:17.130703 32146 consensus_meta.cc:287] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.945s user 0.009s sys 0.008s
I20260807 01:48:17.148003 32140 consensus_queue.cc:260] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:17.148926 32229 consensus_queue.cc:1048] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:48:17.149585 32140 raft_consensus.cc:3060] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:17.150967 27331 raft_consensus.cc:1240] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 5. Current term is 6. Ops: []
W20260807 01:48:17.140743 32053 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260807 01:48:17.143780 27063 leader_election.cc:304] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:17.155344 27063 leader_election.cc:304] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:17.156562 32276 raft_consensus.cc:2749] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:17.157441 27063 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:17.157627 32276 raft_consensus.cc:2749] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260807 01:48:17.160301 32348 log.cc:927] Time spent T c4c4340b503246eca370314d2781034a P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.247s user 0.000s sys 0.001s
W20260807 01:48:17.161265 32071 consensus_meta.cc:287] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.828s user 0.015s sys 0.005s
I20260807 01:48:17.166337 32071 raft_consensus.cc:2749] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:17.174559 27104 raft_consensus.cc:2468] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:48:17.175645 27227 raft_consensus.cc:2393] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 6: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:17.177124 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:17.179709 32347 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:17.180446 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:17.180649 32347 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:17.183054 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 193365us
I20260807 01:48:17.129171 32153 leader_election.cc:290] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:48:17.194656 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 100252us
I20260807 01:48:17.192553 32229 consensus_queue.cc:1048] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.006s
I20260807 01:48:17.199306 27222 raft_consensus.cc:2410] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260807 01:48:17.199250 32229 consensus_queue.cc:1243] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer 57bb4985157447f0a3ab680fa0026f12 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260807 01:48:17.200107 32041 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:17.200505 32249 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:17.200912 32041 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:17.204025 32249 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:17.210625 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:17.212100 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:17.212765 27067 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:17.213810 28714 pending_rounds.cc:85] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12: Aborting all ops after (but not including) 8
I20260807 01:48:17.214267 28714 pending_rounds.cc:107] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260807 01:48:17.214680 31652 raft_consensus.cc:2749] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:17.215678 32347 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:48:17.217214 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 59906.4us
I20260807 01:48:17.218429 27063 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:17.219077 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:17.219882 27213 raft_consensus.cc:2468] T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:48:17.221653 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:17.222411 31652 raft_consensus.cc:2749] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260807 01:48:17.225397 27213 raft_consensus.cc:1240] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260807 01:48:17.189186 32053 raft_consensus.cc:2749] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:48:17.214550 28714 raft_consensus.cc:2889] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 01:48:17.242432 27063 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:17.257035 31203 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:17.259339 31203 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:17.261368 32053 raft_consensus.cc:2749] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 01:48:17.263336 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:17.264721 27330 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
W20260807 01:48:17.261551 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 171579us
I20260807 01:48:17.225847 31960 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:17.268335 32249 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:17.269341 32249 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:17.271927 32249 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:17.272922 28660 leader_election.cc:304] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:17.275630 32146 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:17.261423 27067 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:17.274201 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:17.274899 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 53973.3us
W20260807 01:48:17.275646 27170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:17.276499 32146 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:17.277666 32136 raft_consensus.cc:2749] T 53d8de481420436498aadc9930032da5 P 57bb4985157447f0a3ab680fa0026f12 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260807 01:48:17.278306 27287 leader_election.cc:304] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:17.280597 27287 leader_election.cc:304] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:17.280332 32146 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:17.282537 28655 leader_election.cc:304] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:17.283896 31943 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:17.284857 31652 raft_consensus.cc:2804] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 01:48:17.285434 31652 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:17.285890 31652 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:17.287771 32347 raft_consensus.cc:2804] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:17.288305 32347 raft_consensus.cc:493] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:17.288964 32347 raft_consensus.cc:3060] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
W20260807 01:48:17.293411 32222 consensus_meta.cc:287] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 0.534s user 0.000s sys 0.016s
I20260807 01:48:17.295758 28715 raft_consensus.cc:2393] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 5: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:17.297232 27063 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:17.299286 27063 leader_election.cc:304] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
W20260807 01:48:17.302412 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 50148.1us
I20260807 01:48:17.293602 31898 consensus_queue.cc:1059] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
W20260807 01:48:17.295874 27222 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57662 (request call id 2813) took 4224 ms (client timeout 1824 ms). Trace:
I20260807 01:48:17.288224 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:17.309939 27170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:17.311340 32041 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:17.307845 31771 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:17.313603 27287 leader_election.cc:304] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:17.311337 32326 raft_consensus.cc:2749] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 01:48:17.315232 32229 raft_consensus.cc:2749] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:17.307672 32274 raft_consensus.cc:2749] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260807 01:48:17.308750 27222 rpcz_store.cc:269] 0807 01:48:13.071180 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:13.777135 (+705955us) service_pool.cc:224] Handling call
0807 01:48:17.295586 (+3518451us) spinlock_profiling.cc:243] Waited 3.4 s on lock 0x7b5400125a54. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:17.295765 (+ 179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7829390208}
I20260807 01:48:17.328459 28706 raft_consensus.cc:2468] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:48:17.331283 28711 raft_consensus.cc:1240] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260807 01:48:17.336890 27170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:17.342952 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:48:17.344079 32229 raft_consensus.cc:2804] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:17.345152 32229 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:17.345835 32229 raft_consensus.cc:3060] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:17.349408 27067 leader_election.cc:304] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:17.354595 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:48:17.358803 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 375071us
I20260807 01:48:17.368221 32249 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:17.369288 32249 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:17.371773 32249 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:48:17.358425 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 59252.3us
I20260807 01:48:17.353821 32318 raft_consensus.cc:2804] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election won for term 6
W20260807 01:48:17.384585 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:17.383544 31652 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:17.388870 32352 log.cc:927] Time spent T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.136s user 0.000s sys 0.001s
I20260807 01:48:17.390105 27175 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
W20260807 01:48:17.390806 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 48468us
I20260807 01:48:17.391873 28660 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:17.392493 32362 raft_consensus.cc:2749] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 01:48:17.392956 32262 raft_consensus.cc:2749] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:17.393469 32362 consensus_queue.cc:1059] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260807 01:48:17.394251 32262 consensus_queue.cc:1048] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:48:17.394910 27175 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:48:17.395902 31203 raft_consensus.cc:3055] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 8 LEADER]: Stepping down as leader of term 8
I20260807 01:48:17.396839 32363 raft_consensus.cc:2804] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:48:17.394189 31652 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:17.396270 31203 raft_consensus.cc:740] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:17.399197 27287 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:17.401104 27287 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:17.401870 32139 raft_consensus.cc:2749] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:17.401288 31652 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:17.406311 32222 consensus_queue.cc:1048] T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:48:17.434060 31948 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:17.435495 32136 raft_consensus.cc:3055] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:48:17.438728 32136 raft_consensus.cc:740] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:17.439477 32153 raft_consensus.cc:993] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12: : Instructing follower fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:48:17.440127 32153 raft_consensus.cc:1081] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Signalling peer fcf4e45defb94276bcfa68392711dade to start an election
I20260807 01:48:17.439867 32136 consensus_queue.cc:260] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:17.441323 32136 raft_consensus.cc:3060] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:17.407447 32041 consensus_queue.cc:1048] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 01:48:17.409430 32131 consensus_queue.cc:1048] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:48:17.397549 32363 raft_consensus.cc:697] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [term 6 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:17.406351 27213 raft_consensus.cc:1240] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 7. Current term is 8. Ops: []
I20260807 01:48:17.411854 32130 consensus_queue.cc:1059] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:48:17.409700 31652 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:17.413502 27172 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:17.399201 31203 consensus_queue.cc:260] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:17.415356 32229 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:17.420362 27287 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:17.420950 32139 raft_consensus.cc:2804] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 01:48:17.405009 27063 leader_election.cc:304] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:17.423303 32180 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 12 FOLLOWER]: Advancing to term 13
I20260807 01:48:17.424378 32251 consensus_queue.cc:1048] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260807 01:48:17.425463 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:17.425485 32281 consensus_queue.cc:1059] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260807 01:48:17.427130 32039 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:17.429713 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:48:17.475003 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 80270.4us
I20260807 01:48:17.475806 27105 raft_consensus.cc:2393] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 9: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:17.383709 31771 raft_consensus.cc:2749] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:48:17.470211 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:17.472115 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:17.472162 32139 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:17.481917 32174 raft_consensus.cc:2749] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260807 01:48:17.481621 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 76732.7us
I20260807 01:48:17.483070 27170 leader_election.cc:304] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:17.485334 27063 leader_election.cc:304] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:17.486344 31960 raft_consensus.cc:2804] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:17.486785 31960 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:17.487140 31960 raft_consensus.cc:3060] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:17.487365 32363 consensus_queue.cc:237] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:17.488219 32347 raft_consensus.cc:515] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:17.490939 31896 consensus_queue.cc:1048] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 01:48:17.492378 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:17.493422 27108 raft_consensus.cc:2393] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 5: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:17.492578 27063 leader_election.cc:304] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:17.495664 31652 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:17.495823 32347 leader_election.cc:290] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:17.482738 32229 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:17.498193 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:17.500804 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 87434.6us
I20260807 01:48:17.502661 27172 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:17.503334 27170 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:17.500931 27063 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:17.505441 32326 raft_consensus.cc:2804] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:17.506389 32281 consensus_queue.cc:1243] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer 3262a9d183914b699227fdba0ca5bed3 log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260807 01:48:17.507045 27226 raft_consensus.cc:1240] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 8. Current term is 9. Ops: []
I20260807 01:48:17.507303 32281 raft_consensus.cc:3055] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:48:17.507686 27218 raft_consensus.cc:1275] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260807 01:48:17.507874 32281 raft_consensus.cc:740] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:17.508632 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:17.505949 32326 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:17.509054 32281 consensus_queue.cc:260] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:17.509703 32326 raft_consensus.cc:3060] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:17.503193 32041 raft_consensus.cc:2804] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 01:48:17.516898 28713 raft_consensus.cc:2393] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 9: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:17.486282 31743 raft_consensus.cc:2804] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 01:48:17.449541 32318 raft_consensus.cc:697] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:17.534878 27101 raft_consensus.cc:1275] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260807 01:48:17.477234 32039 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Advancing to term 8
W20260807 01:48:17.536984 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 57887.2us
I20260807 01:48:17.537037 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:17.516585 32139 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Advancing to term 10
W20260807 01:48:17.541450 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 55152.8us
I20260807 01:48:17.543946 27331 raft_consensus.cc:3060] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:17.540647 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:17.517621 32041 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:17.547159 27063 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:17.544767 31953 raft_consensus.cc:2749] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260807 01:48:17.538491 31743 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:48:17.561617 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 141379us
I20260807 01:48:17.503860 27336 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0"
dest_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:36604
I20260807 01:48:17.552911 32041 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:17.603842 27336 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 01:48:17.606467 27336 raft_consensus.cc:3060] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:17.545228 32286 consensus_queue.cc:1048] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260807 01:48:17.605602 28660 leader_election.cc:304] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:17.546877 31960 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:17.602917 31743 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Advancing to term 10
I20260807 01:48:17.616334 27172 leader_election.cc:304] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:17.561487 31908 consensus_queue.cc:1059] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.048s
I20260807 01:48:17.608842 32037 raft_consensus.cc:3060] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:17.626170 31908 consensus_queue.cc:1243] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer 57bb4985157447f0a3ab680fa0026f12 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260807 01:48:17.628618 32356 log.cc:927] Time spent T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.352s user 0.003s sys 0.001s
W20260807 01:48:17.633708 32110 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.999s user 0.008s sys 0.014s
W20260807 01:48:17.634393 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 131170us
I20260807 01:48:17.638672 32318 consensus_queue.cc:237] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:17.692651 27336 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:17.640825 32281 raft_consensus.cc:3060] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:17.686602 31652 raft_consensus.cc:2804] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260807 01:48:17.688074 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 127750us
I20260807 01:48:17.688400 27064 leader_election.cc:304] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:17.689158 32347 raft_consensus.cc:2749] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260807 01:48:17.689756 32110 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.161s user 0.009s sys 0.014s
I20260807 01:48:17.566795 32222 raft_consensus.cc:2749] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260807 01:48:17.691632 31960 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:48:17.640067 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 82008.5us
I20260807 01:48:17.694437 32041 raft_consensus.cc:515] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:17.702850 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 98263.5us
W20260807 01:48:17.686775 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 151475us
W20260807 01:48:17.724936 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 50178.9us
I20260807 01:48:17.702389 31652 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
W20260807 01:48:17.709301 32377 log.cc:927] Time spent T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.094s user 0.000s sys 0.000s
I20260807 01:48:17.710610 32037 raft_consensus.cc:2749] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:17.711876 31771 raft_consensus.cc:3060] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:17.713562 32071 consensus_queue.cc:1059] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 01:48:17.714548 32351 consensus_queue.cc:1048] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:48:17.737782 31948 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:17.738370 27170 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:17.738601 31948 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:17.739352 32222 raft_consensus.cc:2749] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260807 01:48:17.740063 27170 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:17.740850 32222 raft_consensus.cc:2749] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:17.740985 31948 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:17.726501 31652 raft_consensus.cc:3060] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:17.742276 32222 consensus_queue.cc:1048] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 01:48:17.746840 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:17.723374 27336 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:17.708308 31203 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:17.728865 27331 raft_consensus.cc:2468] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:48:17.732523 31908 raft_consensus.cc:2804] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader election won for term 9
I20260807 01:48:17.732877 27172 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:17.735903 32071 raft_consensus.cc:3055] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 01:48:17.763640 27214 raft_consensus.cc:1275] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260807 01:48:17.730033 32053 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:17.777837 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:17.761451 32071 raft_consensus.cc:740] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:17.770129 31960 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:17.795468 27099 raft_consensus.cc:1240] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 4. Current term is 5. Ops: []
I20260807 01:48:17.775408 31908 raft_consensus.cc:697] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [term 9 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:17.762406 32347 consensus_queue.cc:1048] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
W20260807 01:48:17.801151 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:17.803457 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:17.793884 32131 raft_consensus.cc:2804] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:48:17.780930 32053 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:17.813767 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:17.802537 32130 raft_consensus.cc:493] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:17.852676 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:17.813519 32131 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:17.859390 27099 raft_consensus.cc:2468] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:17.818948 27170 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:48:17.814424 32351 consensus_queue.cc:1048] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260807 01:48:17.833463 27103 raft_consensus.cc:1275] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260807 01:48:17.802106 32135 consensus_queue.cc:1048] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260807 01:48:17.884126 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:17.809137 31960 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:17.895674 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:17.892980 27101 raft_consensus.cc:2468] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:48:17.875185 32131 raft_consensus.cc:3060] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:17.904606 27230 raft_consensus.cc:2410] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260807 01:48:17.901156 32043 raft_consensus.cc:2749] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260807 01:48:17.929623 27174 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:17.931015 32388 log.cc:927] Time spent T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.128s user 0.001s sys 0.001s
W20260807 01:48:17.931452 32379 log.cc:927] Time spent T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.203s user 0.000s sys 0.001s
I20260807 01:48:17.879935 32130 raft_consensus.cc:515] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:17.932838 32380 log.cc:927] Time spent T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.085s user 0.000s sys 0.000s
I20260807 01:48:17.932595 32326 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:17.936015 28655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:17.948581 31203 raft_consensus.cc:2804] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader election won for term 9
I20260807 01:48:17.949386 28655 leader_election.cc:304] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:17.938199 27103 raft_consensus.cc:1240] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260807 01:48:17.861370 32163 consensus_queue.cc:1059] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.168s
W20260807 01:48:17.931774 32378 log.cc:927] Time spent T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.230s user 0.000s sys 0.001s
I20260807 01:48:17.951388 32163 raft_consensus.cc:3055] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:48:17.951809 32163 raft_consensus.cc:740] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:17.952857 32163 consensus_queue.cc:260] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:17.954417 32163 raft_consensus.cc:3060] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:17.956065 32326 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:17.959167 32347 raft_consensus.cc:2804] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:48:17.959728 32347 raft_consensus.cc:697] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 6 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
W20260807 01:48:17.959980 32136 consensus_meta.cc:287] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 0.518s user 0.013s sys 0.000s
I20260807 01:48:17.961663 27099 raft_consensus.cc:1240] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 6. Current term is 7. Ops: [6.5-6.5]
W20260807 01:48:17.963173 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 199545us
I20260807 01:48:17.953521 32139 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:17.907656 32071 consensus_queue.cc:260] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:17.966137 32053 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:17.967994 32130 leader_election.cc:290] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:17.939594 32041 leader_election.cc:290] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:17.978542 31203 raft_consensus.cc:697] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [term 9 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
W20260807 01:48:17.985438 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 193763us
I20260807 01:48:17.997857 27170 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:17.983497 31953 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:17.973896 32039 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:18.011998 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:18.009842 31203 consensus_queue.cc:237] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:18.006845 31953 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:18.014293 31771 raft_consensus.cc:2749] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:17.944875 31908 consensus_queue.cc:237] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:18.024137 27230 raft_consensus.cc:3060] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:18.045390 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:18.055238 27170 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
W20260807 01:48:18.060132 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 110832us
I20260807 01:48:17.989704 31652 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:18.066854 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:18.040911 32180 consensus_meta.cc:287] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 0.519s user 0.011s sys 0.000s
I20260807 01:48:18.011912 32041 raft_consensus.cc:2804] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election won for term 5
I20260807 01:48:18.067453 28702 raft_consensus.cc:2468] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:18.075789 28714 raft_consensus.cc:1240] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 4. Current term is 5. Ops: []
I20260807 01:48:18.070860 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:18.086994 27102 raft_consensus.cc:1240] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 7. Current term is 8. Ops: []
I20260807 01:48:18.087536 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:18.099049 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:18.099812 28715 raft_consensus.cc:3055] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:48:18.100153 28715 raft_consensus.cc:740] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:18.101049 28715 consensus_queue.cc:260] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:18.061609 31771 raft_consensus.cc:493] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:18.102052 28715 raft_consensus.cc:3060] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
W20260807 01:48:18.106630 27063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:18.071012 32126 raft_consensus.cc:3055] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 7 LEADER]: Stepping down as leader of term 7
W20260807 01:48:18.086102 32357 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.827s user 0.023s sys 0.024s
I20260807 01:48:18.081823 32135 raft_consensus.cc:2804] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 01:48:18.109812 28706 raft_consensus.cc:1217] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 8 LEADER]: Deduplicated request from leader. Original: 4.6->[7.7-7.7] Dedup: 7.7->[]
I20260807 01:48:18.022109 31743 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:18.120882 28706 raft_consensus.cc:1240] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 8 LEADER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260807 01:48:18.131357 32222 raft_consensus.cc:493] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:18.132117 32222 raft_consensus.cc:515] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:18.130512 27230 raft_consensus.cc:2468] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:48:18.122946 32135 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:18.135057 32135 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Advancing to term 10
W20260807 01:48:18.135298 32396 log.cc:927] Time spent T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.092s user 0.000s sys 0.000s
W20260807 01:48:18.120738 32357 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.858s user 0.024s sys 0.025s
W20260807 01:48:18.144307 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 89181us
I20260807 01:48:18.107378 32362 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:18.145030 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
W20260807 01:48:18.131759 32399 log.cc:927] Time spent T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.122s user 0.001s sys 0.000s
I20260807 01:48:18.117476 32126 raft_consensus.cc:740] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:18.036373 32347 consensus_queue.cc:237] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:18.131659 32053 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:18.156890 27170 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:48:18.118661 32186 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:18.118269 32041 raft_consensus.cc:697] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:18.129320 31960 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:18.122581 31771 raft_consensus.cc:515] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:18.158221 32136 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:18.205947 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:18.189108 32281 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:18.193789 32039 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:18.201932 32222 leader_election.cc:290] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:18.228979 31203 raft_consensus.cc:2804] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:18.229820 31203 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
W20260807 01:48:18.232863 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 76081.2us
I20260807 01:48:18.230556 31203 raft_consensus.cc:3060] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:18.192414 32362 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:18.214601 31743 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:18.190712 32186 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:18.254091 28715 raft_consensus.cc:2468] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:48:18.194832 32053 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:18.215263 32281 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:18.205106 32136 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:18.242882 28711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48362 (request call id 876) took 1083 ms. Trace:
I20260807 01:48:18.238602 27170 leader_election.cc:304] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
W20260807 01:48:18.256986 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 95649.6us
I20260807 01:48:18.260767 27175 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:18.258968 28711 rpcz_store.cc:276] 0807 01:48:17.159386 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:17.159555 (+ 169us) service_pool.cc:224] Handling call
0807 01:48:17.329577 (+170022us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:17.331771 (+ 2194us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:17.557250 (+225479us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x7b540056c7d4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:18.242590 (+685340us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b540056c7d0. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541dc8d 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:18.242770 (+ 180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":393327744}
I20260807 01:48:18.275746 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:18.276811 27218 raft_consensus.cc:2393] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 9: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:18.278730 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:18.280447 32110 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:18.281137 32110 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:18.285141 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:18.281088 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 14 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:18.287561 32425 raft_consensus.cc:2749] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 01:48:18.228652 32222 consensus_queue.cc:1059] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260807 01:48:18.290118 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:18.290885 27222 raft_consensus.cc:2468] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:48:18.288188 32110 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:18.271826 31743 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:18.294390 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 167600us
I20260807 01:48:18.297137 27218 raft_consensus.cc:2410] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260807 01:48:18.285912 31652 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:18.299727 27230 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 8.
W20260807 01:48:18.305794 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 71448.3us
I20260807 01:48:18.307375 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:18.264427 32126 consensus_queue.cc:260] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:18.297180 32425 raft_consensus.cc:2749] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:48:18.306344 31948 raft_consensus.cc:493] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:18.274151 32041 consensus_queue.cc:237] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:18.322049 27170 leader_election.cc:304] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:18.328260 31652 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:18.328564 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:18.329015 31652 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:18.329769 32186 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:18.331441 32053 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:18.332237 31652 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:18.335708 32174 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:18.336252 32174 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:18.337966 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d235fc7e09b8465f8ddc570370605464" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:18.338822 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:18.339241 27103 raft_consensus.cc:1240] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 5. Current term is 6. Ops: []
I20260807 01:48:18.339552 27109 raft_consensus.cc:2376] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5. Re-sending same reply.
I20260807 01:48:18.339967 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:18.314479 32222 raft_consensus.cc:3055] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Stepping down as leader of term 4
I20260807 01:48:18.339495 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:18.343345 32222 raft_consensus.cc:740] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:18.343528 32174 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:18.328300 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:18.344578 32222 consensus_queue.cc:260] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:18.345628 27104 raft_consensus.cc:1275] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260807 01:48:18.347080 32281 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:18.347874 27330 raft_consensus.cc:2468] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
I20260807 01:48:18.349256 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:18.349992 27104 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 8.
I20260807 01:48:18.360953 32362 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:18.362129 32136 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:18.363631 27338 raft_consensus.cc:1275] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260807 01:48:18.361299 27099 raft_consensus.cc:1240] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 6. Current term is 7. Ops: []
I20260807 01:48:18.364655 32071 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:18.321233 31771 leader_election.cc:290] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:18.317134 31743 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:18.373660 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:18.374840 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:18.373782 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:18.346086 32222 raft_consensus.cc:3060] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:18.392850 31203 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:18.397347 27100 raft_consensus.cc:2393] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 6: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:18.393733 27104 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 9.
I20260807 01:48:18.316882 32363 consensus_queue.cc:1059] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260807 01:48:18.399670 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:18.360255 32416 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:18.405501 31203 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:18.406314 32416 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:18.406767 31743 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:18.406996 31953 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:18.408658 32416 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:18.409197 32135 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:18.409520 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:18.410431 27101 raft_consensus.cc:3060] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:18.414307 32428 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:18.415112 32428 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:18.419642 32135 leader_election.cc:290] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:18.420825 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:18.421685 27326 raft_consensus.cc:2393] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 6: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:18.422087 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:18.422538 32363 consensus_queue.cc:1243] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Peer 57bb4985157447f0a3ab680fa0026f12 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260807 01:48:18.422819 27327 raft_consensus.cc:3060] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:18.424916 32139 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:18.426144 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 112086us
I20260807 01:48:18.427042 27170 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:48:18.428619 27170 leader_election.cc:304] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:18.429895 32131 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:18.432101 32131 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:18.432876 27327 raft_consensus.cc:2468] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
I20260807 01:48:18.434937 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:18.435596 27326 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 8.
I20260807 01:48:18.436597 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:18.437009 27287 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:18.437220 27326 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 8.
I20260807 01:48:18.443228 27216 raft_consensus.cc:1240] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 7. Current term is 8. Ops: []
I20260807 01:48:18.444568 27101 raft_consensus.cc:2468] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:48:18.445652 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:18.446942 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:18.447700 27327 raft_consensus.cc:2468] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 9.
I20260807 01:48:18.448652 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:48:18.451208 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 154654us
I20260807 01:48:18.453210 31203 raft_consensus.cc:2749] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:48:18.449915 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:18.460655 27338 raft_consensus.cc:1275] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260807 01:48:18.462879 27327 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 13.
I20260807 01:48:18.462795 32428 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:20.047794 27327 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33976 (request call id 2725) took 1611 ms (client timeout 1937 ms). Trace:
I20260807 01:48:18.465348 32425 raft_consensus.cc:2804] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260807 01:48:20.048393 27327 rpcz_store.cc:269] 0807 01:48:18.436260 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:18.449603 (+ 13343us) service_pool.cc:224] Handling call
0807 01:48:20.047645 (+1598042us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:20.048830 32425 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:20.049317 32425 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Advancing to term 9
W20260807 01:48:18.537153 32386 log.cc:927] Time spent T d235fc7e09b8465f8ddc570370605464 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.904s user 0.007s sys 0.001s
W20260807 01:48:18.687894 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:20.052043 27064 connection.cc:256] RPC call timeout handler was delayed by -1.1015963461250067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17270000000000002s for a response
I20260807 01:48:20.050143 27327 raft_consensus.cc:1275] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260807 01:48:19.621906 27331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33976 (request call id 2705) took 1304 ms. Trace:
I20260807 01:48:19.813347 27337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33976 (request call id 2713) took 1407 ms. Trace:
I20260807 01:48:20.053340 27327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33976 (request call id 2726) took 1616 ms. Trace:
I20260807 01:48:19.954447 32317 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:18.457535 32139 raft_consensus.cc:2804] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260807 01:48:19.965195 32041 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.566s user 0.014s sys 0.006s
I20260807 01:48:20.054694 27327 rpcz_store.cc:276] 0807 01:48:18.436614 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:20.049447 (+1612833us) service_pool.cc:224] Handling call
0807 01:48:20.049810 (+ 363us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:20.052988 (+ 3178us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:20.053269 (+ 281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:20.001421 31908 raft_consensus.cc:493] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:20.004127 32456 raft_consensus.cc:493] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:48:20.010761 27103 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.662s user 0.003s sys 0.051s
I20260807 01:48:20.056340 31908 raft_consensus.cc:515] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } attrs { replace: true } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:20.056509 32456 raft_consensus.cc:515] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:20.057986 27103 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.710s user 0.003s sys 0.052s
I20260807 01:48:20.058852 27103 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1829) took 1712 ms. Trace:
I20260807 01:48:18.404877 32362 raft_consensus.cc:2749] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 01:48:20.019403 32401 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:20.059144 31908 leader_election.cc:290] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:20.055855 32041 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.657s user 0.014s sys 0.007s
I20260807 01:48:20.059877 31908 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:20.060390 31908 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:20.026481 32415 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:20.059689 32401 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:20.061396 32415 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:20.062036 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:20.054962 32139 raft_consensus.cc:493] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:20.059124 32456 leader_election.cc:290] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:20.063159 32139 raft_consensus.cc:3060] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:20.052512 32459 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:20.053542 27337 rpcz_store.cc:276] 0807 01:48:18.405493 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:18.405727 (+ 234us) service_pool.cc:224] Handling call
0807 01:48:18.460340 (+ 54613us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:19.712508 (+1252168us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:19.712545 (+ 37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:19.713107 (+ 562us) log.cc:844] Serialized 26 byte log entry
0807 01:48:19.811969 (+ 98862us) raft_consensus.cc:1645] Marking committed up to 2
0807 01:48:19.812016 (+ 47us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:19.812093 (+ 77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:19.813069 (+ 976us) raft_consensus.cc:1685] finished
0807 01:48:19.813093 (+ 24us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:19.813270 (+ 177us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":98403,"threads_started":1,"wal-append.queue_time_us":98014}
I20260807 01:48:18.405115 31948 raft_consensus.cc:515] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:20.059191 27103 rpcz_store.cc:276] 0807 01:48:18.346859 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:18.347067 (+ 208us) service_pool.cc:224] Handling call
0807 01:48:18.347389 (+ 322us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:18.347659 (+ 270us) raft_consensus.cc:1534] Early marking committed up to index 11
0807 01:48:18.348014 (+ 355us) log.cc:844] Serialized 14 byte log entry
0807 01:48:20.058478 (+1710464us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:20.058536 (+ 58us) raft_consensus.cc:1645] Marking committed up to 11
0807 01:48:20.058587 (+ 51us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:20.058657 (+ 70us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:20.058767 (+ 110us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":732,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"thread_start_us":1710072,"threads_started":1,"wal-append.queue_time_us":162452,"wal-append.run_cpu_time_us":2041,"wal-append.run_wall_time_us":1002029}
I20260807 01:48:20.053109 27331 rpcz_store.cc:276] 0807 01:48:18.317353 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:18.317504 (+ 151us) service_pool.cc:224] Handling call
0807 01:48:18.328357 (+ 10853us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:18.328726 (+ 369us) raft_consensus.cc:1534] Early marking committed up to index 5
0807 01:48:18.329061 (+ 335us) log.cc:844] Serialized 14 byte log entry
0807 01:48:19.621194 (+1292133us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:19.621300 (+ 106us) raft_consensus.cc:1645] Marking committed up to 5
0807 01:48:19.621386 (+ 86us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:19.621500 (+ 114us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:19.621737 (+ 237us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":5411,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":48,"raft.queue_time_us":1164991,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":219,"spinlock_wait_cycles":95872,"thread_start_us":126647,"threads_started":1,"wal-append.queue_time_us":202631,"wal-append.run_cpu_time_us":1472,"wal-append.run_wall_time_us":1001481}
I20260807 01:48:20.060781 32362 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:20.060338 32041 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:20.065584 31948 leader_election.cc:290] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:48:20.021781 27325 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.596s user 0.021s sys 0.026s
I20260807 01:48:20.066107 32362 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:20.066680 32459 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:20.066797 27325 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.641s user 0.022s sys 0.026s
I20260807 01:48:20.069098 32459 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:20.069298 27325 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33976 (request call id 2711) took 1707 ms. Trace:
I20260807 01:48:20.068465 27064 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:20.069859 27325 rpcz_store.cc:276] 0807 01:48:18.362237 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:18.403000 (+ 40763us) service_pool.cc:224] Handling call
0807 01:48:18.424330 (+ 21330us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:18.424640 (+ 310us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:18.424664 (+ 24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:18.425182 (+ 518us) log.cc:844] Serialized 26 byte log entry
0807 01:48:20.068950 (+1643768us) raft_consensus.cc:1645] Marking committed up to 7
0807 01:48:20.068988 (+ 38us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:20.069054 (+ 66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:20.069095 (+ 41us) raft_consensus.cc:1685] finished
0807 01:48:20.069105 (+ 10us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:20.069237 (+ 132us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1643341,"threads_started":1,"wal-append.queue_time_us":107038,"wal-append.run_cpu_time_us":1670,"wal-append.run_wall_time_us":1001726}
I20260807 01:48:20.062718 27109 raft_consensus.cc:2468] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:48:20.071797 32362 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:20.063277 32401 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:20.063545 32415 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:20.072420 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:20.083163 27109 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:20.083776 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:20.085141 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:20.085078 32425 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:20.086059 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:20.073025 32043 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:20.087288 32043 raft_consensus.cc:515] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:20.087723 32425 leader_election.cc:290] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:20.087951 32415 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:20.073511 32317 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:20.073779 31960 raft_consensus.cc:493] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:20.088469 32415 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:20.088799 31960 raft_consensus.cc:515] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:20.062677 31908 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:20.066129 32041 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:20.079706 27174 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:20.090643 32415 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:48:20.091001 27174 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:20.091907 32041 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:20.092447 32041 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:20.093255 27218 raft_consensus.cc:1275] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260807 01:48:20.080958 31896 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:20.093963 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:20.094206 31908 raft_consensus.cc:493] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:20.094451 31896 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:20.095253 32037 raft_consensus.cc:493] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:20.095445 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:20.095819 32037 raft_consensus.cc:515] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:20.096344 31652 raft_consensus.cc:493] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:48:20.039026 32071 consensus_meta.cc:287] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 1.674s user 0.017s sys 0.009s
I20260807 01:48:20.092969 32041 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:20.097419 32071 raft_consensus.cc:2749] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260807 01:48:20.095033 31908 raft_consensus.cc:515] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:20.099004 32041 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:20.100484 31908 leader_election.cc:290] T 73e9ae0228aa41a996315efe07c04299 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:20.100742 31896 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:20.096921 31652 raft_consensus.cc:515] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:20.091012 31960 leader_election.cc:290] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:20.103974 27103 raft_consensus.cc:1240] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 7. Current term is 8. Ops: []
I20260807 01:48:20.104557 31652 leader_election.cc:290] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:20.106734 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:20.106894 32037 leader_election.cc:290] T c4c4340b503246eca370314d2781034a P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:20.107811 27222 raft_consensus.cc:1240] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 8. Current term is 9. Ops: []
I20260807 01:48:20.089962 32317 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:20.086742 27101 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
I20260807 01:48:20.110646 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:20.110934 27338 raft_consensus.cc:3055] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:48:20.111495 27218 raft_consensus.cc:2468] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 13.
I20260807 01:48:20.110672 27103 raft_consensus.cc:1240] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 4. Current term is 5. Ops: []
I20260807 01:48:20.113224 27218 raft_consensus.cc:1240] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 10 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 9. Current term is 10. Ops: []
W20260807 01:48:20.114952 27333 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.695s user 0.009s sys 0.027s
W20260807 01:48:20.115506 27333 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.696s user 0.010s sys 0.027s
I20260807 01:48:20.116272 27333 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33976 (request call id 2714) took 1700 ms. Trace:
I20260807 01:48:20.116608 27333 rpcz_store.cc:276] 0807 01:48:18.415779 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:18.417683 (+ 1904us) service_pool.cc:224] Handling call
0807 01:48:18.418123 (+ 440us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:18.418409 (+ 286us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:18.418435 (+ 26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:18.419133 (+ 698us) log.cc:844] Serialized 26 byte log entry
0807 01:48:20.115940 (+1696807us) raft_consensus.cc:1645] Marking committed up to 5
0807 01:48:20.115980 (+ 40us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:20.116049 (+ 69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:20.116087 (+ 38us) raft_consensus.cc:1685] finished
0807 01:48:20.116098 (+ 11us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:20.116219 (+ 121us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1696349,"threads_started":1,"wal-append.queue_time_us":119199,"wal-append.run_cpu_time_us":2031,"wal-append.run_wall_time_us":1002064}
I20260807 01:48:20.117404 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:20.118064 27218 raft_consensus.cc:2468] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:48:20.118525 27172 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:20.119711 27338 raft_consensus.cc:740] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 5 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:20.120405 27172 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:18.405125 27099 raft_consensus.cc:1275] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260807 01:48:20.121609 27099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34468 (request call id 1833) took 1754 ms. Trace:
I20260807 01:48:20.122282 31908 raft_consensus.cc:2804] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:20.122803 31908 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:20.123252 31908 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:20.121974 27099 rpcz_store.cc:276] 0807 01:48:18.367051 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:18.367328 (+ 277us) service_pool.cc:224] Handling call
0807 01:48:18.395498 (+ 28170us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:20.121331 (+1725833us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:20.121551 (+ 220us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 01:48:20.125144 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 1.69665e+06us
I20260807 01:48:20.126091 27170 leader_election.cc:304] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:20.130347 27218 raft_consensus.cc:1240] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Rejecting Update request from peer 3262a9d183914b699227fdba0ca5bed3 for earlier term 6. Current term is 7. Ops: []
I20260807 01:48:20.132961 32318 consensus_queue.cc:1048] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 01:48:20.135062 32041 raft_consensus.cc:2804] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:48:20.135526 32041 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:20.135897 32041 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Advancing to term 8
W20260807 01:48:20.136094 27330 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.772s user 0.034s sys 0.025s
W20260807 01:48:20.136711 27330 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.773s user 0.034s sys 0.025s
I20260807 01:48:20.137684 27330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33976 (request call id 2709) took 1777 ms. Trace:
I20260807 01:48:20.138187 27170 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
W20260807 01:48:20.140069 27063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:20.140520 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:20.141209 27226 raft_consensus.cc:2468] T 00395257beb442e892a4ec18c4c30d95 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 8.
I20260807 01:48:20.143616 27170 leader_election.cc:304] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:20.167404 32146 raft_consensus.cc:2749] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260807 01:48:20.138098 27330 rpcz_store.cc:276] 0807 01:48:18.359971 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:18.360157 (+ 186us) service_pool.cc:224] Handling call
0807 01:48:18.362675 (+ 2518us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:18.362990 (+ 315us) raft_consensus.cc:1534] Early marking committed up to index 4
0807 01:48:18.363339 (+ 349us) log.cc:844] Serialized 14 byte log entry
0807 01:48:20.137287 (+1773948us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:20.137347 (+ 60us) raft_consensus.cc:1645] Marking committed up to 4
0807 01:48:20.137398 (+ 51us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:20.137478 (+ 80us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:20.137619 (+ 141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3110,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":441,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"thread_start_us":1773401,"threads_started":1,"wal-append.queue_time_us":168402,"wal-append.run_cpu_time_us":1048,"wal-append.run_wall_time_us":1002912}
I20260807 01:48:20.147367 31953 consensus_queue.cc:1059] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 01:48:20.145996 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:20.168689 32419 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:20.169184 27105 raft_consensus.cc:2393] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 7: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:20.169786 27105 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 1826) took 1834 ms. Trace:
I20260807 01:48:20.169396 32419 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:20.143538 27328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33976 (request call id 2716) took 1722 ms. Trace:
I20260807 01:48:20.149267 32037 raft_consensus.cc:2749] T 2cfefe1ff8f14fb68678ac128e129d32 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:20.120719 27338 consensus_queue.cc:260] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:20.151000 32131 raft_consensus.cc:2804] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 13 FOLLOWER]: Leader election won for term 13
I20260807 01:48:20.139254 32318 consensus_queue.cc:1059] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260807 01:48:20.154588 31203 consensus_queue.cc:1059] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260807 01:48:20.156428 28655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:20.157070 32473 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:20.162696 32174 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:20.189291 32473 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:20.080726 32139 raft_consensus.cc:515] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:20.148834 31896 consensus_queue.cc:1059] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:48:20.165624 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 14 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:20.156910 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:20.170122 27105 rpcz_store.cc:276] 0807 01:48:18.335522 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:18.338293 (+ 2771us) service_pool.cc:224] Handling call
0807 01:48:20.145818 (+1807525us) spinlock_profiling.cc:243] Waited 5.92 ms on lock 0x7b5800024c98. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d522e0521 00007f7d522e03d2 00007f7d5b24c219 00007f7d5a39f640 00007f7d5a467ebb 00007f7d5a467f41 00007f7d5a415eb0 00007f7d5a415794 00007f7d5395e834 00007f7d5395e7c2
0807 01:48:20.169693 (+ 23875us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13623680}
I20260807 01:48:20.170370 27328 rpcz_store.cc:276] 0807 01:48:18.420520 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:18.420715 (+ 195us) service_pool.cc:224] Handling call
0807 01:48:18.421071 (+ 356us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:20.141500 (+1720429us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:20.141528 (+ 28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:20.141576 (+ 48us) raft_consensus.cc:1645] Marking committed up to 3
0807 01:48:20.141616 (+ 40us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:20.143206 (+ 1590us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b5400021cd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:20.143258 (+ 52us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:20.143379 (+ 121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3956053248}
I20260807 01:48:20.191452 27338 raft_consensus.cc:3060] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:18.456130 27289 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
W20260807 01:48:20.214905 27336 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.872s user 0.011s sys 0.019s
W20260807 01:48:21.796489 27336 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.453s user 0.012s sys 0.019s
I20260807 01:48:20.194573 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:21.798040 28707 raft_consensus.cc:2393] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 7: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
W20260807 01:48:21.798558 28707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52322 (request call id 783) took 1609 ms (client timeout 1705 ms). Trace:
W20260807 01:48:21.798908 28707 rpcz_store.cc:269] 0807 01:48:20.188792 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:20.193948 (+ 5156us) service_pool.cc:224] Handling call
0807 01:48:21.798473 (+1604525us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:21.800113 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:21.800804 28707 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 9.
I20260807 01:48:20.195245 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:21.802924 28711 raft_consensus.cc:2468] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 4.
I20260807 01:48:21.803447 28711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52322 (request call id 784) took 1612 ms. Trace:
I20260807 01:48:21.803730 28711 rpcz_store.cc:276] 0807 01:48:20.191310 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:20.191435 (+ 125us) service_pool.cc:224] Handling call
0807 01:48:21.803389 (+1611954us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:21.804652 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:21.805248 28711 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:48:20.224553 32479 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:20.244403 32482 raft_consensus.cc:493] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:48:20.262842 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:21.806516 32479 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:21.806629 32482 raft_consensus.cc:515] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:21.808938 32479 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:21.809062 32482 leader_election.cc:290] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:21.809552 32479 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:21.809813 32482 raft_consensus.cc:2804] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:48:21.810443 32482 raft_consensus.cc:697] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
W20260807 01:48:21.197690 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 1.05405e+06us
W20260807 01:48:21.463831 27174 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:21.485842 27288 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.317s user 0.015s sys 0.028s
W20260807 01:48:21.812002 27288 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.644s user 0.015s sys 0.029s
I20260807 01:48:21.812179 27170 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:21.529557 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:21.815805 27229 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Advancing to term 8
W20260807 01:48:21.566237 27109 consensus_meta.cc:287] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3: Time spent flushing consensus metadata: real 1.482s user 0.017s sys 0.014s
I20260807 01:48:21.816747 27109 raft_consensus.cc:2468] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 9.
W20260807 01:48:21.817286 27109 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41080 (request call id 3144) took 1746 ms (client timeout 1675 ms). Trace:
W20260807 01:48:21.817629 27109 rpcz_store.cc:269] 0807 01:48:20.071159 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:20.072157 (+ 998us) service_pool.cc:224] Handling call
0807 01:48:21.817174 (+1745017us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:20.169227 31948 raft_consensus.cc:493] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:21.801966 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:21.821172 27337 raft_consensus.cc:1240] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 9. Current term is 10. Ops: []
I20260807 01:48:21.821167 31948 raft_consensus.cc:515] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:20.168687 31953 raft_consensus.cc:3055] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:48:21.824035 31953 raft_consensus.cc:740] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:21.825098 31953 consensus_queue.cc:260] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:20.193437 32131 raft_consensus.cc:697] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 13 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:21.828666 32131 consensus_queue.cc:237] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:21.829273 27328 raft_consensus.cc:1240] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 5. Current term is 6. Ops: []
I20260807 01:48:21.524693 32163 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:21.831025 32163 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:21.833057 32163 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:48:21.443917 32222 consensus_meta.cc:287] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f: Time spent flushing consensus metadata: real 3.046s user 0.014s sys 0.021s
I20260807 01:48:21.835398 32222 raft_consensus.cc:493] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:21.835978 32222 raft_consensus.cc:515] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:20.203557 32180 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:20.238842 31908 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:21.839883 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:21.840473 27214 raft_consensus.cc:3060] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:21.841578 32222 leader_election.cc:290] T 08edf043c99f445b99e723211e8e0ad0 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:20.206049 32037 raft_consensus.cc:3055] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 6 LEADER]: Stepping down as leader of term 6
I20260807 01:48:21.842983 32037 raft_consensus.cc:740] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:21.843708 31948 leader_election.cc:290] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:21.844022 32037 consensus_queue.cc:260] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:21.845178 31953 raft_consensus.cc:3060] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:21.845299 32037 raft_consensus.cc:3060] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:20.261616 32041 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:20.501761 32126 raft_consensus.cc:3060] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:21.848325 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:21.849232 32041 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:20.194252 32139 leader_election.cc:290] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:21.850562 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:20.189416 32174 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:21.855895 27214 raft_consensus.cc:2468] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:48:21.857447 32174 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:21.523624 32347 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.157s user 0.043s sys 0.019s
W20260807 01:48:21.860358 32347 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.494s user 0.043s sys 0.022s
I20260807 01:48:20.196198 32419 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:20.169341 32428 raft_consensus.cc:493] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:21.863214 32428 raft_consensus.cc:515] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:21.865437 32428 leader_election.cc:290] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:20.003784 32458 raft_consensus.cc:493] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:21.866917 32458 raft_consensus.cc:515] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:21.869165 32458 leader_election.cc:290] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:20.250190 32473 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:21.592255 27326 consensus_meta.cc:287] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade: Time spent flushing consensus metadata: real 3.145s user 0.023s sys 0.030s
I20260807 01:48:21.870776 27326 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
I20260807 01:48:21.595010 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:21.872071 28710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52322 (request call id 785) took 1656 ms (client timeout 1877 ms). Trace:
W20260807 01:48:21.872459 28710 rpcz_store.cc:269] 0807 01:48:20.215670 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:21.594656 (+1378986us) service_pool.cc:224] Handling call
0807 01:48:21.871997 (+277341us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:21.873472 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:21.606191 32493 raft_consensus.cc:493] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:48:21.874504 32493 raft_consensus.cc:515] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:21.876807 32493 leader_election.cc:290] T 53d8de481420436498aadc9930032da5 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:21.616397 32410 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:21.628389 32497 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:48:21.878527 32497 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:21.636392 32499 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:21.880229 32499 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:21.640931 27222 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.509s user 0.023s sys 0.017s
I20260807 01:48:21.644758 32500 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:21.880721 31953 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:21.882453 32500 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:21.882887 31953 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:21.884869 32500 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:21.877575 32410 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:21.871373 27326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33976 (request call id 2721) took 3436 ms (client timeout 1705 ms). Trace:
W20260807 01:48:21.886690 27326 rpcz_store.cc:269] 0807 01:48:18.435127 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:18.437969 (+ 2842us) service_pool.cc:224] Handling call
0807 01:48:21.871267 (+3433298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:21.888195 27326 raft_consensus.cc:1240] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 9. Current term is 10. Ops: []
W20260807 01:48:21.890419 27222 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.758s user 0.026s sys 0.017s
I20260807 01:48:21.892627 31953 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:21.895087 32499 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:21.897083 32410 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:21.897823 27213 raft_consensus.cc:2468] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
W20260807 01:48:21.899818 27213 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57662 (request call id 2875) took 1793 ms (client timeout 2255 ms). Trace:
W20260807 01:48:21.900221 27213 rpcz_store.cc:269] 0807 01:48:20.106343 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:20.106456 (+ 113us) service_pool.cc:224] Handling call
0807 01:48:21.899625 (+1793169us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b54001f6dd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d554226da 00007f7d5a415927 00007f7d5395e834 00007f7d5395e7c2 00007f7d5395e6f1 00007f7d5395e66c 00007f7d5395d922 00007f7d5375807c 00007f7d537574b6 00007f7d53756e5f
0807 01:48:21.899754 (+ 129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4117390592}
I20260807 01:48:21.901612 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:21.902901 27222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57662 (request call id 2882) took 1788 ms. Trace:
I20260807 01:48:21.903976 32497 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:48:21.525844 27063 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.548s (SENT) [suppressed 20 similar messages]
I20260807 01:48:21.679134 27227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57662 (request call id 2859) took 4173 ms. Trace:
I20260807 01:48:21.922551 27227 rpcz_store.cc:276] 0807 01:48:17.505324 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:17.511149 (+ 5825us) service_pool.cc:224] Handling call
0807 01:48:17.551488 (+ 40339us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:17.631128 (+ 79640us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 01:48:17.631159 (+ 31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:17.631781 (+ 622us) log.cc:844] Serialized 26 byte log entry
0807 01:48:17.880145 (+248364us) raft_consensus.cc:1645] Marking committed up to 5
0807 01:48:17.880188 (+ 43us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:21.678670 (+3798482us) spinlock_profiling.cc:243] Waited 66.3 ms on lock 0x7b54001de054. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:21.678730 (+ 60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:21.678781 (+ 51us) raft_consensus.cc:1685] finished
0807 01:48:21.678793 (+ 12us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:21.679038 (+ 245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":152376576,"thread_start_us":247970,"threads_started":1,"wal-append.queue_time_us":342612,"wal-append.run_cpu_time_us":20978,"wal-append.run_wall_time_us":1955240}
I20260807 01:48:21.924669 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:21.681017 26937 maintenance_manager.cc:419] P 31e8851a2fee40aa9ea5af43f94890df: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033458
W20260807 01:48:21.710263 27003 debug-util.cc:398] Leaking SignalData structure 0x7b080018a580 after lost signal to thread 24600
W20260807 01:48:21.928963 27003 debug-util.cc:398] Leaking SignalData structure 0x7b0800043ea0 after lost signal to thread 27011
W20260807 01:48:21.929373 27003 debug-util.cc:398] Leaking SignalData structure 0x7b08000c7d40 after lost signal to thread 27036
W20260807 01:48:21.713181 28660 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:21.930603 27003 debug-util.cc:398] Leaking SignalData structure 0x7b0800353880 after lost signal to thread 27146
W20260807 01:48:21.930936 27003 debug-util.cc:398] Leaking SignalData structure 0x7b08001d6340 after lost signal to thread 27148
I20260807 01:48:21.716200 27216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57654 (request call id 1965) took 3442 ms. Trace:
W20260807 01:48:21.932116 27003 debug-util.cc:398] Leaking SignalData structure 0x7b080035cca0 after lost signal to thread 27269
W20260807 01:48:21.932475 27003 debug-util.cc:398] Leaking SignalData structure 0x7b08002c3d00 after lost signal to thread 27270
I20260807 01:48:21.719847 32270 raft_consensus.cc:493] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:21.725128 27067 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.557s user 0.008s sys 0.010s
I20260807 01:48:21.931658 27216 rpcz_store.cc:276] 0807 01:48:18.273691 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:18.273822 (+ 131us) service_pool.cc:224] Handling call
0807 01:48:18.289756 (+ 15934us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:18.443792 (+154036us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:21.715874 (+3272082us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b54001f7cd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:21.716121 (+ 247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":352264960}
W20260807 01:48:21.726948 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 3.2898e+06us
W20260807 01:48:21.933624 27003 debug-util.cc:398] Leaking SignalData structure 0x7b08002f7de0 after lost signal to thread 27378
I20260807 01:48:21.734345 32492 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:21.935143 27003 debug-util.cc:398] Leaking SignalData structure 0x7b08003ea2a0 after lost signal to thread 28748
I20260807 01:48:21.934752 32492 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:21.935488 27003 debug-util.cc:398] Leaking SignalData structure 0x7b0800377900 after lost signal to thread 28750
W20260807 01:48:21.936514 27003 debug-util.cc:398] Leaking SignalData structure 0x7b08003a89c0 after lost signal to thread 32304
W20260807 01:48:21.936913 27003 debug-util.cc:398] Leaking SignalData structure 0x7b08001eb4c0 after lost signal to thread 32351
W20260807 01:48:21.937122 27003 debug-util.cc:398] Leaking SignalData structure 0x7b080030b900 after lost signal to thread 32352
I20260807 01:48:21.736121 27104 raft_consensus.cc:1240] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 5. Current term is 6. Ops: []
W20260807 01:48:21.748495 27102 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.591s user 0.000s sys 0.019s
W20260807 01:48:21.937902 27102 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.781s user 0.000s sys 0.020s
I20260807 01:48:21.758921 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:21.938529 28715 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 8.
I20260807 01:48:21.932987 32270 raft_consensus.cc:515] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:21.938946 32524 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
W20260807 01:48:21.938621 28660 connection.cc:256] RPC call timeout handler was delayed by -2.748447204940021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17090000000000002s for a response
W20260807 01:48:21.940306 28660 connection.cc:256] RPC call timeout handler was delayed by -2.5585472050588578s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.192s for a response
W20260807 01:48:21.940586 28660 connection.cc:256] RPC call timeout handler was delayed by -2.5430758269503713s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16100000000000003s for a response
W20260807 01:48:21.941057 28660 connection.cc:256] RPC call timeout handler was delayed by -2.47754720505327s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20099999999999998s for a response
I20260807 01:48:21.761380 28713 raft_consensus.cc:1275] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260807 01:48:21.938112 27104 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 738) took 1877 ms. Trace:
W20260807 01:48:21.932978 27067 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.765s user 0.009s sys 0.010s
I20260807 01:48:21.761999 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:21.770625 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:21.772225 28709 raft_consensus.cc:1275] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260807 01:48:21.778374 32110 raft_consensus.cc:493] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:21.779498 27100 raft_consensus.cc:2410] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260807 01:48:21.947768 32497 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:48:21.780673 27212 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 9.
I20260807 01:48:21.948423 32497 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:21.778995 28655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:21.948982 27212 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57662 (request call id 2872) took 1855 ms (client timeout 1614 ms). Trace:
I20260807 01:48:21.948496 27100 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34468 (request call id 1837) took 3543 ms. Trace:
I20260807 01:48:21.951431 27100 rpcz_store.cc:276] 0807 01:48:18.405433 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:20.165115 (+1759682us) service_pool.cc:224] Handling call
0807 01:48:21.948390 (+1783275us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 01:48:21.950471 27212 rpcz_store.cc:269] 0807 01:48:20.093804 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:20.156522 (+ 62718us) service_pool.cc:224] Handling call
0807 01:48:21.948896 (+1792374us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:21.953891 27212 raft_consensus.cc:1240] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 8. Current term is 9. Ops: []
W20260807 01:48:21.789049 27064 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.638s user 0.021s sys 0.025s
W20260807 01:48:21.957767 27064 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.807s user 0.021s sys 0.025s
W20260807 01:48:21.958124 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 1.80749e+06us
I20260807 01:48:21.958788 27064 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:21.959975 27064 leader_election.cc:304] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:21.960701 32493 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:21.961216 32493 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:21.961519 27064 leader_election.cc:304] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:20.267282 32483 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:21.818384 32130 raft_consensus.cc:2749] T aff03a52fbfd498ab6d04a579e96fd0e P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 01:48:21.810060 32479 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:21.962565 32483 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:21.963503 32493 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:21.806105 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:21.964848 28711 raft_consensus.cc:2393] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 6: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:21.819518 27175 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:21.965293 32497 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:21.965649 31652 consensus_queue.cc:1059] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 01:48:21.966301 27213 raft_consensus.cc:2410] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260807 01:48:20.244956 32318 raft_consensus.cc:493] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:21.967608 32479 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:21.968263 32318 raft_consensus.cc:515] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:21.969287 32479 raft_consensus.cc:2804] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:21.969738 32479 raft_consensus.cc:493] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:21.838743 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:21.970105 32479 raft_consensus.cc:3060] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
W20260807 01:48:21.839535 27379 heartbeater.cc:646] Failed to heartbeat to 127.24.4.62:35633 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.24.4.62:35633 timed out after 15.000s (SENT)
I20260807 01:48:21.839448 32180 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:20.252009 31960 raft_consensus.cc:3055] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 8 LEADER]: Stepping down as leader of term 8
I20260807 01:48:21.839905 32363 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:21.972514 31960 raft_consensus.cc:740] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:21.972720 32363 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:21.973630 31960 consensus_queue.cc:260] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:21.975145 31960 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:21.975875 32363 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:21.976035 32483 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:21.851583 27327 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 8.
I20260807 01:48:21.852722 27333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:21.977391 27333 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:48:21.835330 32482 consensus_queue.cc:237] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:21.842926 31908 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:21.854074 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:21.979092 27330 raft_consensus.cc:2410] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260807 01:48:21.859122 27229 raft_consensus.cc:2468] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 8.
W20260807 01:48:21.980157 27229 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57662 (request call id 2873) took 1886 ms (client timeout 1998 ms). Trace:
W20260807 01:48:21.980581 27229 rpcz_store.cc:269] 0807 01:48:20.094068 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:21.529020 (+1434952us) service_pool.cc:224] Handling call
0807 01:48:21.980089 (+451069us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:21.859711 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:21.862270 27338 raft_consensus.cc:1275] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260807 01:48:21.982198 27338 pending_rounds.cc:85] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade: Aborting all ops after (but not including) 6
I20260807 01:48:21.982497 27338 pending_rounds.cc:107] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260807 01:48:21.982728 27338 raft_consensus.cc:2889] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 01:48:21.983083 32479 raft_consensus.cc:515] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:21.984038 27230 raft_consensus.cc:1240] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 7. Current term is 8. Ops: []
I20260807 01:48:21.985448 32479 leader_election.cc:290] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:21.871272 32416 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:21.986625 32416 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:21.989696 32416 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:21.993125 31948 raft_consensus.cc:493] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:21.862496 28708 raft_consensus.cc:1240] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [term 6 LEADER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6. Ops: []
I20260807 01:48:21.995429 28708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48350 (request call id 955) took 3952 ms. Trace:
I20260807 01:48:21.995787 28708 rpcz_store.cc:276] 0807 01:48:18.043536 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:18.043662 (+ 126us) service_pool.cc:224] Handling call
0807 01:48:18.054703 (+ 11041us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:21.994551 (+3939848us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:21.995121 (+ 570us) spinlock_profiling.cc:243] Waited 3.81 s on lock 0x7b5400577bd4. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d55387951 00007f7d5541f1e1 00007f7d5541dc1e 00007f7d5a414229 00007f7d53959d34 00007f7d53959cc2 00007f7d53959bf1 00007f7d53959b6c 00007f7d53958e22 00007f7d5375807c 00007f7d537574b6
0807 01:48:21.995336 (+ 215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8756407040}
I20260807 01:48:21.993798 31948 raft_consensus.cc:515] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:21.998768 26950 catalog_manager.cc:5697] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 3 to 4, leader changed from 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4) to fcf4e45defb94276bcfa68392711dade (127.24.4.3). New cstate: current_term: 4 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 01:48:22.000068 31948 leader_election.cc:290] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:21.973446 32180 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:21.966601 31652 consensus_queue.cc:1059] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.036s
I20260807 01:48:22.003547 31652 raft_consensus.cc:3055] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 01:48:22.003921 31652 raft_consensus.cc:740] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:22.004767 31652 consensus_queue.cc:260] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:21.905663 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:22.007599 28710 raft_consensus.cc:2468] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:48:21.903331 27222 rpcz_store.cc:276] 0807 01:48:20.114687 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:20.119365 (+ 4678us) service_pool.cc:224] Handling call
0807 01:48:20.131631 (+ 12266us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:20.131872 (+ 241us) raft_consensus.cc:1534] Early marking committed up to index 5
0807 01:48:21.890860 (+1758988us) log.cc:844] Serialized 14 byte log entry
0807 01:48:21.902515 (+ 11655us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:21.902572 (+ 57us) raft_consensus.cc:1645] Marking committed up to 5
0807 01:48:21.902623 (+ 51us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:21.902689 (+ 66us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:21.902818 (+ 129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":133008,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":778,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":167,"thread_start_us":1769834,"threads_started":2,"wal-append.queue_time_us":7915}
I20260807 01:48:21.910087 32425 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:22.009980 32425 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.013548 27216 raft_consensus.cc:1275] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260807 01:48:22.012907 32425 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:22.021375 32318 leader_election.cc:290] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:21.654326 32501 raft_consensus.cc:493] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:22.026902 32501 raft_consensus.cc:515] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.031401 32501 leader_election.cc:290] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:22.035115 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 73640.2us
I20260807 01:48:22.036376 27064 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:22.037482 27064 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12
W20260807 01:48:21.916476 26952 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46138 (request call id 399) took 14051 ms (client timeout 15000 ms). Trace:
I20260807 01:48:21.939646 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:21.760281 28706 raft_consensus.cc:1275] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260807 01:48:21.939495 32524 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:21.941390 28660 connection.cc:256] RPC call timeout handler was delayed by -2.3164472049102187s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2189s for a response
I20260807 01:48:21.945818 32163 raft_consensus.cc:493] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:21.933816 27287 leader_election.cc:304] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:21.946341 32251 raft_consensus.cc:493] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:21.946682 32347 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:21.857492 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:48:21.947191 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 421375us
I20260807 01:48:21.942714 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:21.954339 32410 raft_consensus.cc:493] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:48:21.941218 32270 leader_election.cc:290] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:21.946008 32110 raft_consensus.cc:515] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:21.937399 32492 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:21.955133 32503 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:21.946262 27104 rpcz_store.cc:276] 0807 01:48:20.060443 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:21.735599 (+1675156us) service_pool.cc:224] Handling call
0807 01:48:21.735949 (+ 350us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:21.937837 (+201888us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:21.938046 (+ 209us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 01:48:21.965873 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 146439us
I20260807 01:48:21.966414 32130 raft_consensus.cc:2804] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 01:48:21.821753 28707 raft_consensus.cc:2468] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:21.788311 32043 leader_election.cc:290] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:21.965905 32497 raft_consensus.cc:2804] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:48:21.974298 32473 raft_consensus.cc:3055] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 9 LEADER]: Stepping down as leader of term 9
I20260807 01:48:21.850301 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:21.850544 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:21.966928 32493 raft_consensus.cc:2804] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260807 01:48:21.983496 27338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33976 (request call id 2722) took 3548 ms. Trace:
I20260807 01:48:22.039428 32500 raft_consensus.cc:2804] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 01:48:21.847330 27336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33976 (request call id 2704) took 3533 ms. Trace:
W20260807 01:48:21.914434 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 3.45812e+06us
W20260807 01:48:21.937439 27003 debug-util.cc:398] Leaking SignalData structure 0x7b0800329980 after lost signal to thread 32366
I20260807 01:48:22.169735 32347 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.171211 27212 raft_consensus.cc:3055] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:48:22.171701 27212 raft_consensus.cc:740] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:22.172354 32347 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:22.172535 27212 consensus_queue.cc:260] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.173867 27212 raft_consensus.cc:3060] T bf07cc968f3343869d12c10e81ce75f9 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:22.174532 27226 raft_consensus.cc:2410] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260807 01:48:22.175318 28713 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:22.177141 32130 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:22.177768 32130 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Advancing to term 10
I20260807 01:48:22.180857 32503 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:22.182569 32497 raft_consensus.cc:493] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:22.183073 32497 raft_consensus.cc:3060] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:22.183483 32503 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:48:22.040015 26952 rpcz_store.cc:269] 0807 01:48:07.864241 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:07.864434 (+ 193us) service_pool.cc:224] Handling call
0807 01:48:21.915867 (+14051433us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 01:48:16.149855 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 01:48:16.152217 (+ 2362us) write_op.cc:432] Acquiring schema lock in shared mode
0807 01:48:16.152274 (+ 57us) write_op.cc:435] Acquired schema lock
0807 01:48:16.152292 (+ 18us) tablet.cc:662] Decoding operations
0807 01:48:16.184727 (+ 32435us) write_op.cc:620] Acquiring the partition lock for write op
0807 01:48:16.200750 (+ 16023us) write_op.cc:641] Partition lock acquired for write op
0807 01:48:16.208286 (+ 7536us) tablet.cc:685] Acquiring locks for 12 operations
0807 01:48:16.479320 (+271034us) tablet.cc:701] Row locks acquired
0807 01:48:16.497629 (+ 18309us) write_op.cc:260] PREPARE: finished
0807 01:48:16.515657 (+ 18028us) write_op.cc:270] Start()
0807 01:48:16.529000 (+ 13343us) write_op.cc:276] Timestamp: P: 1786067296509126 usec, L: 0
0807 01:48:16.536601 (+ 7601us) op_driver.cc:348] REPLICATION: starting
0807 01:48:16.722312 (+185711us) log.cc:844] Serialized 4727 byte log entry
0807 01:48:17.258932 (+536620us) op_driver.cc:464] REPLICATION: finished
0807 01:48:17.949521 (+690589us) spinlock_profiling.cc:243] Waited 71.2 ms on lock 0x7b440002250c. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d521a0981 00007f7d5223e855 00007f7d5223f272 00007f7d54fc93bb 00007f7d54fc4c9f 00007f7d522faa5c 00007f7d5b1a2fc9 00007f7d52316fdb 00007f7d523152db 00007f7d52317542 00007f7d58e748d0
0807 01:48:21.799262 (+3849741us) spinlock_profiling.cc:243] Waited 1.97 ms on lock 0x7b180041e3c0. stack: 00007f7d522e5cf3 00007f7d522e5bf0 00007f7d51c70574 00007f7d5ba066c4 00007f7d5ba06659 00007f7d5b233f31 00007f7d5b2406d1 00007f7d52317d33 00007f7d5231a121 00007f7d5231a0d9 00007f7d5231a069 00007f7d5231a031 00007f7d5231932d 00000000003aa554 00000000003aa399 00007f7d522fb1ea
0807 01:48:21.799462 (+ 200us) write_op.cc:301] APPLY: starting
0807 01:48:21.799578 (+ 116us) tablet.cc:1367] starting BulkCheckPresence
0807 01:48:21.799927 (+ 349us) tablet.cc:1370] finished BulkCheckPresence
0807 01:48:21.799967 (+ 40us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 01:48:21.801556 (+ 1589us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 01:48:21.801957 (+ 401us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0807 01:48:21.802020 (+ 63us) write_op.cc:312] APPLY: finished
0807 01:48:21.808469 (+ 6449us) log.cc:844] Serialized 89 byte log entry
0807 01:48:21.871823 (+ 63354us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:48:21.872084 (+ 261us) write_op.cc:454] Released schema lock
0807 01:48:21.872150 (+ 66us) write_op.cc:341] FINISH: Updating metrics
0807 01:48:21.875871 (+ 3721us) write_op.cc:489] Releasing partition, row and schema locks
0807 01:48:21.875899 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":896590,"apply.run_cpu_time_us":8413,"apply.run_wall_time_us":76369,"num_ops":12,"prepare.queue_time_us":404312,"prepare.run_cpu_time_us":25808,"prepare.run_wall_time_us":925137,"raft.queue_time_us":157797,"raft.run_cpu_time_us":49344,"raft.run_wall_time_us":897920,"replication_time_us":722073,"spinlock_wait_cycles":168304768,"thread_start_us":1615580,"threads_started":5,"wal-append.queue_time_us":238558,"wal-append.run_cpu_time_us":12481,"wal-append.run_wall_time_us":1330846}]]}
I20260807 01:48:22.185317 32493 raft_consensus.cc:493] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:22.185884 32493 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 13 FOLLOWER]: Advancing to term 14
I20260807 01:48:22.043928 32524 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:22.192852 27218 raft_consensus.cc:2410] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260807 01:48:22.192660 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:22.189474 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:22.193614 28715 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 7.
I20260807 01:48:22.194494 31948 raft_consensus.cc:2749] T aff03a52fbfd498ab6d04a579e96fd0e P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 01:48:22.190246 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:22.006023 31652 raft_consensus.cc:3060] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:22.195907 28711 raft_consensus.cc:2468] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:48:22.174256 27336 rpcz_store.cc:276] 0807 01:48:18.316935 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:18.317099 (+ 164us) service_pool.cc:224] Handling call
0807 01:48:18.321515 (+ 4416us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:18.342214 (+ 20699us) raft_consensus.cc:1534] Early marking committed up to index 3
0807 01:48:18.342717 (+ 503us) log.cc:844] Serialized 14 byte log entry
0807 01:48:21.846935 (+3504218us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 01:48:21.846992 (+ 57us) raft_consensus.cc:1645] Marking committed up to 3
0807 01:48:21.847042 (+ 50us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:21.847106 (+ 64us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:21.847237 (+ 131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2690,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":65,"raft.queue_time_us":47715,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"thread_start_us":3503635,"threads_started":2,"wal-append.queue_time_us":163967,"wal-append.run_cpu_time_us":1827,"wal-append.run_wall_time_us":1001847}
I20260807 01:48:22.186265 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:22.198005 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.198208 28706 raft_consensus.cc:2410] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260807 01:48:22.198680 27226 raft_consensus.cc:2468] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:48:22.198918 32110 leader_election.cc:290] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:22.186317 32473 raft_consensus.cc:740] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:22.001217 26950 catalog_manager.cc:5697] T 53d8de481420436498aadc9930032da5 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 01:48:22.200734 32473 consensus_queue.cc:260] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:22.202113 27325 raft_consensus.cc:1240] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 5. Current term is 6. Ops: []
W20260807 01:48:22.195611 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 261889us
I20260807 01:48:22.169941 27328 raft_consensus.cc:2468] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:48:22.208611 27287 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:22.202971 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.210584 27226 raft_consensus.cc:2468] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:48:22.212337 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.213148 27226 raft_consensus.cc:2393] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 7: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:22.214304 32473 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Advancing to term 10
I20260807 01:48:22.212352 32482 raft_consensus.cc:2749] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 01:48:22.216447 27287 leader_election.cc:304] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:22.204804 32251 raft_consensus.cc:515] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.186596 27337 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
I20260807 01:48:22.193815 32500 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:22.205556 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.224442 32500 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Advancing to term 10
I20260807 01:48:22.224624 27216 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
I20260807 01:48:22.193522 27338 rpcz_store.cc:276] 0807 01:48:18.435407 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:20.110502 (+1675095us) service_pool.cc:224] Handling call
0807 01:48:20.110797 (+ 295us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 01:48:21.983200 (+1872403us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:21.983432 (+ 232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 01:48:22.041332 32318 raft_consensus.cc:2804] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader election won for term 8
W20260807 01:48:22.113372 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 75955.5us
I20260807 01:48:22.227921 27287 leader_election.cc:304] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:22.228796 31948 raft_consensus.cc:2749] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:22.229534 31948 raft_consensus.cc:2804] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:48:22.229871 31948 raft_consensus.cc:493] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:22.230329 31948 raft_consensus.cc:3060] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:22.203737 26950 catalog_manager.cc:5697] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "fcf4e45defb94276bcfa68392711dade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } health_report { overall_health: HEALTHY } } }
I20260807 01:48:22.203297 32410 raft_consensus.cc:515] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.195889 32163 raft_consensus.cc:515] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.240914 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:48:22.202795 28655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:22.222231 32478 raft_consensus.cc:493] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
W20260807 01:48:22.227094 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:22.184466 27175 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:22.234035 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.236171 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:22.243009 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:22.243150 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:48:22.238235 27064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:22.239228 27102 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52556 (request call id 736) took 2179 ms. Trace:
W20260807 01:48:22.244066 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 59693.2us
I20260807 01:48:22.219079 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:22.243548 27213 raft_consensus.cc:2393] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 8: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:22.245100 27175 leader_election.cc:304] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:22.245496 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:22.243960 27064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:22.246438 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.247233 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.246382 28702 raft_consensus.cc:2410] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 6 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260807 01:48:22.250057 27229 raft_consensus.cc:2393] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 8: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:22.246661 32478 raft_consensus.cc:515] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.251066 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.251724 27229 raft_consensus.cc:2468] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:48:22.247047 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.253398 27230 raft_consensus.cc:2410] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260807 01:48:22.254297 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:22.254518 32483 raft_consensus.cc:493] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:22.254920 28711 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 7.
I20260807 01:48:22.247421 32482 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:22.247852 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.256285 27214 raft_consensus.cc:2468] T 565d0a4463ac45e4bad1886cc0095ff7 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 9.
I20260807 01:48:22.243862 27227 raft_consensus.cc:2468] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
W20260807 01:48:22.248704 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:22.257154 27287 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:22.257570 32482 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:22.258610 27287 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:48:22.259001 32416 raft_consensus.cc:2749] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:22.259686 32482 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:22.259907 32416 raft_consensus.cc:2804] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:22.260499 32416 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:22.260756 32497 raft_consensus.cc:515] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.261065 32126 raft_consensus.cc:2749] T 8b00b70a17a64e288e95507deb091fc1 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:22.255060 32483 raft_consensus.cc:515] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:22.259907 27287 leader_election.cc:304] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:22.262599 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.263054 32482 raft_consensus.cc:2749] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:22.263274 27226 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 8.
I20260807 01:48:22.263468 32483 leader_election.cc:290] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:22.264233 32478 leader_election.cc:290] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:22.264160 32126 consensus_queue.cc:1048] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:48:22.266408 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:22.267143 27213 raft_consensus.cc:3060] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:22.267951 27175 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:22.244055 27102 rpcz_store.cc:276] 0807 01:48:20.059827 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 01:48:20.156345 (+ 96518us) service_pool.cc:224] Handling call
0807 01:48:20.156770 (+ 425us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 01:48:21.940381 (+1783611us) raft_consensus.cc:1534] Early marking committed up to index 6
0807 01:48:21.940413 (+ 32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 01:48:21.940949 (+ 536us) log.cc:844] Serialized 26 byte log entry
0807 01:48:22.222027 (+281078us) raft_consensus.cc:1645] Marking committed up to 6
0807 01:48:22.222072 (+ 45us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 01:48:22.222183 (+ 111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 01:48:22.238976 (+ 16793us) raft_consensus.cc:1685] finished
0807 01:48:22.239003 (+ 27us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 01:48:22.239157 (+ 154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85672,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"thread_start_us":2061708,"threads_started":2,"wal-append.queue_time_us":279499}
I20260807 01:48:22.269429 32126 raft_consensus.cc:2804] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:22.269884 32126 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:22.270401 32126 raft_consensus.cc:3060] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:22.271867 32043 raft_consensus.cc:493] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:22.255898 32163 leader_election.cc:290] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:22.273099 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.273317 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:22.274010 27222 raft_consensus.cc:2393] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 8: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
W20260807 01:48:22.244392 27063 leader_election.cc:336] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.912s (SENT)
I20260807 01:48:22.255693 32492 raft_consensus.cc:2804] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:48:22.260875 32416 raft_consensus.cc:3060] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:22.274135 28707 raft_consensus.cc:2468] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:48:22.249183 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.279666 27212 raft_consensus.cc:2468] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:48:22.263510 27287 leader_election.cc:304] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:22.283497 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:22.275568 27175 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:22.287894 27063 leader_election.cc:304] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:22.290834 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:22.293031 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:22.293634 28713 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:22.294504 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:22.284497 32163 consensus_queue.cc:1048] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260807 01:48:22.291544 27337 raft_consensus.cc:2468] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:48:22.241605 32318 raft_consensus.cc:697] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [term 8 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:22.296917 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:22.297392 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:22.297556 27289 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:22.284408 32270 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:22.276081 32492 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:22.298539 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:22.299639 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:22.301216 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:22.301220 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:22.308879 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:22.301815 27325 raft_consensus.cc:1275] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Refusing update from remote peer 57bb4985157447f0a3ab680fa0026f12: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260807 01:48:22.327445 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:22.320881 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.315506 27289 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:22.328945 27109 raft_consensus.cc:2468] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:48:22.331331 27289 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:22.332123 32419 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:22.332412 32139 raft_consensus.cc:2804] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:48:22.332940 32139 raft_consensus.cc:493] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:22.332860 32419 raft_consensus.cc:515] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:22.333470 32139 raft_consensus.cc:3060] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:22.324851 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:22.335278 32419 leader_election.cc:290] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:48:22.311676 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 48269.4us
I20260807 01:48:22.336370 27287 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:22.337003 32419 raft_consensus.cc:2749] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 01:48:22.325909 28715 raft_consensus.cc:2468] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 5.
I20260807 01:48:22.326623 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.326689 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:22.307224 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:22.320184 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.337770 32419 consensus_queue.cc:1059] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260807 01:48:22.340144 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:22.351365 27100 raft_consensus.cc:2468] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:48:22.351599 32511 raft_consensus.cc:3055] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 7 LEADER]: Stepping down as leader of term 7
I20260807 01:48:22.352031 32511 raft_consensus.cc:740] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:22.298194 32510 raft_consensus.cc:493] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:22.348712 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.353048 27222 raft_consensus.cc:2410] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260807 01:48:22.328464 28706 raft_consensus.cc:2468] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:48:22.338621 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.354239 27104 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 8.
I20260807 01:48:22.325594 28709 raft_consensus.cc:2410] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260807 01:48:22.298372 31908 raft_consensus.cc:2804] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:22.357857 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.358748 27102 raft_consensus.cc:2468] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
W20260807 01:48:22.355221 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:22.321489 32492 raft_consensus.cc:3060] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:22.337956 27287 leader_election.cc:304] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:22.277266 32043 raft_consensus.cc:515] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.362115 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:48:22.357775 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 122975us
I20260807 01:48:22.353477 27105 raft_consensus.cc:2468] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 3.
I20260807 01:48:22.364197 27289 leader_election.cc:304] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:22.324474 32503 raft_consensus.cc:493] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:22.301383 32512 raft_consensus.cc:2804] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:22.324798 32163 raft_consensus.cc:2749] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:22.298269 32483 raft_consensus.cc:493] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:22.321578 32428 raft_consensus.cc:2804] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:22.322681 32251 leader_election.cc:290] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:22.374954 32497 leader_election.cc:290] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:22.325018 32473 consensus_queue.cc:1048] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260807 01:48:22.389748 27213 raft_consensus.cc:2468] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:48:22.379077 32318 consensus_queue.cc:237] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.308552 32410 leader_election.cc:290] T 54eb75662df043488517f09bff1dfcff P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:22.373224 27108 raft_consensus.cc:1275] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260807 01:48:22.381685 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:22.384334 27109 raft_consensus.cc:2468] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:48:22.386831 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.387713 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:22.400761 28711 raft_consensus.cc:2468] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:48:22.358425 31908 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:22.389021 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:22.361086 32510 raft_consensus.cc:515] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:22.391853 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:22.404647 28709 raft_consensus.cc:3060] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Advancing to term 6
I20260807 01:48:22.392330 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:22.325296 32270 raft_consensus.cc:515] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.389885 32482 raft_consensus.cc:2804] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:22.391986 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:22.396322 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:22.405336 27102 raft_consensus.cc:3060] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:22.392513 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:22.407215 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 85920.8us
I20260807 01:48:22.380720 32512 raft_consensus.cc:493] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:22.408386 28707 raft_consensus.cc:2393] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 7: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
I20260807 01:48:22.411634 28714 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 8.
I20260807 01:48:22.393622 27175 leader_election.cc:304] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:22.386376 32428 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:22.370934 32524 consensus_queue.cc:1059] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260807 01:48:22.418326 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.404331 31908 raft_consensus.cc:3060] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:22.419257 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:22.422977 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:22.419727 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 56068.5us
W20260807 01:48:22.424000 27063 leader_election.cc:336] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 2.137s (SENT)
I20260807 01:48:22.424461 27063 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:22.389192 32483 raft_consensus.cc:515] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:22.428015 27063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:22.385907 32503 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.391144 32130 raft_consensus.cc:515] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.413686 32482 raft_consensus.cc:493] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:22.432418 28713 raft_consensus.cc:2468] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:48:22.436825 32503 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:22.428562 28660 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:22.417650 32428 raft_consensus.cc:3060] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:22.416484 32512 raft_consensus.cc:3060] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:22.443120 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.410470 32493 raft_consensus.cc:515] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:22.450799 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:22.432235 32497 raft_consensus.cc:2749] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260807 01:48:22.456312 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 118426us
I20260807 01:48:22.461102 32126 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.451893 28660 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:22.419060 32410 consensus_queue.cc:1048] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260807 01:48:22.432699 32458 raft_consensus.cc:2764] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260807 01:48:22.494803 28713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:22.468781 27287 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:22.470501 28709 raft_consensus.cc:2468] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:48:22.422043 32251 consensus_queue.cc:1059] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260807 01:48:22.500398 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.501206 27109 raft_consensus.cc:2468] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:22.443919 32482 raft_consensus.cc:3060] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:22.492769 27108 raft_consensus.cc:1275] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260807 01:48:22.493876 27175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 100318us
W20260807 01:48:22.497714 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 45916.1us
I20260807 01:48:22.432968 32500 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:22.504441 28660 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:22.505571 32110 raft_consensus.cc:2804] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:22.505729 28709 raft_consensus.cc:1275] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 01:48:22.488366 32538 raft_consensus.cc:2804] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:22.516155 32541 raft_consensus.cc:2749] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 01:48:22.516528 32043 leader_election.cc:290] T 51cef821f0284d8e9369e98d38f80e73 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:22.527606 32500 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:22.531380 32126 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:22.534264 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.533689 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:22.538218 27108 raft_consensus.cc:2393] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 10: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:22.538457 27102 raft_consensus.cc:2468] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8.
I20260807 01:48:22.509857 32110 raft_consensus.cc:493] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:22.543210 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.545817 27102 raft_consensus.cc:2393] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 6: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:22.548624 32524 consensus_queue.cc:1048] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:48:22.409687 32511 consensus_queue.cc:260] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:22.515513 32251 raft_consensus.cc:3055] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 8 LEADER]: Stepping down as leader of term 8
I20260807 01:48:22.514086 32538 raft_consensus.cc:493] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:22.532665 32270 leader_election.cc:290] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:22.548695 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:22.505837 31948 raft_consensus.cc:515] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.571971 28709 raft_consensus.cc:3060] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:22.572540 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:22.575016 27102 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:22.568953 32251 raft_consensus.cc:740] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:22.551923 32110 raft_consensus.cc:3060] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:22.576143 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
W20260807 01:48:22.577675 28742 debug-util.cc:398] Leaking SignalData structure 0x7b080022b780 after lost signal to thread 24600
I20260807 01:48:22.553468 32130 leader_election.cc:290] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:22.553241 32545 raft_consensus.cc:2804] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 01:48:22.581871 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:48:22.586449 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 224608us
I20260807 01:48:22.588663 27108 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:22.571024 32538 raft_consensus.cc:3060] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:22.583841 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:22.584827 28742 debug-util.cc:398] Leaking SignalData structure 0x7b08002df7a0 after lost signal to thread 27011
W20260807 01:48:22.593086 28742 debug-util.cc:398] Leaking SignalData structure 0x7b0800286700 after lost signal to thread 27036
I20260807 01:48:22.593001 28707 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 9.
I20260807 01:48:22.575280 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:22.594491 28714 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Advancing to term 10
W20260807 01:48:22.585024 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 116350us
I20260807 01:48:22.595850 27287 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
W20260807 01:48:22.595901 28742 debug-util.cc:398] Leaking SignalData structure 0x7b08002f18e0 after lost signal to thread 27146
I20260807 01:48:22.581194 27109 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 9.
I20260807 01:48:22.532765 32139 raft_consensus.cc:515] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:22.599085 28742 debug-util.cc:398] Leaking SignalData structure 0x7b08003041a0 after lost signal to thread 27148
W20260807 01:48:22.600483 28742 debug-util.cc:398] Leaking SignalData structure 0x7b08002235c0 after lost signal to thread 27269
I20260807 01:48:22.600220 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51cef821f0284d8e9369e98d38f80e73" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
W20260807 01:48:22.600777 28742 debug-util.cc:398] Leaking SignalData structure 0x7b08002fb200 after lost signal to thread 27270
W20260807 01:48:22.601742 28742 debug-util.cc:398] Leaking SignalData structure 0x7b08003ab1c0 after lost signal to thread 27378
W20260807 01:48:22.601935 28742 debug-util.cc:398] Leaking SignalData structure 0x7b080035b7c0 after lost signal to thread 27379
I20260807 01:48:22.605082 32139 leader_election.cc:290] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:22.606823 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.608163 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.608750 27110 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 9.
I20260807 01:48:22.609653 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:22.610682 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.583428 32546 consensus_queue.cc:1048] T bf9ccfe7f7e64481b739c6efef03b95a P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260807 01:48:22.611341 27099 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8.
I20260807 01:48:22.612282 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.613075 27099 raft_consensus.cc:2410] T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260807 01:48:22.619077 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "879666f362ba4a2db3fdfe6738b37dec" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:22.605877 32139 raft_consensus.cc:2804] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:48:22.619824 27101 raft_consensus.cc:2376] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 9. Re-sending same reply.
I20260807 01:48:22.606609 32541 raft_consensus.cc:2804] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 01:48:22.588456 32545 raft_consensus.cc:493] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:22.620657 27110 raft_consensus.cc:2376] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 9. Re-sending same reply.
I20260807 01:48:22.623471 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.623876 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.624615 27109 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8.
W20260807 01:48:22.625820 28742 debug-util.cc:398] Leaking SignalData structure 0x7b080027d940 after lost signal to thread 28748
W20260807 01:48:22.626277 28742 debug-util.cc:398] Leaking SignalData structure 0x7b0800169540 after lost signal to thread 28750
I20260807 01:48:22.632164 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.633107 27104 raft_consensus.cc:2468] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:22.633358 28709 raft_consensus.cc:2468] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:22.621622 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.635159 27100 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 9.
I20260807 01:48:22.639068 28714 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 10.
W20260807 01:48:22.638595 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 91397.5us
W20260807 01:48:22.621263 32535 log.cc:927] Time spent T 89dc6a602e9e42399d533f67d4bd9bef P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.238s user 0.004s sys 0.000s
I20260807 01:48:22.639420 32483 leader_election.cc:290] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
W20260807 01:48:22.650653 32543 log.cc:927] Time spent T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.051s user 0.001s sys 0.000s
I20260807 01:48:22.651047 32510 leader_election.cc:290] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:22.651886 32493 leader_election.cc:290] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 14 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:22.621373 32529 raft_consensus.cc:3055] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 01:48:22.652789 32529 raft_consensus.cc:740] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:22.639235 32545 raft_consensus.cc:3060] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:22.653890 32529 consensus_queue.cc:260] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.655330 32529 raft_consensus.cc:3060] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:22.656358 32511 raft_consensus.cc:3060] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:22.657627 32139 raft_consensus.cc:697] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [term 6 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:22.658716 32139 consensus_queue.cc:237] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.659180 32538 raft_consensus.cc:515] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.661460 32538 leader_election.cc:290] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:22.664621 32416 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:22.666783 32416 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:22.668017 32428 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.669114 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.672732 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.673559 27101 raft_consensus.cc:2468] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:22.675920 27105 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 10.
I20260807 01:48:22.682485 27175 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:22.685634 27102 raft_consensus.cc:2468] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8.
I20260807 01:48:22.686942 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.688956 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.686623 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:22.690071 27100 raft_consensus.cc:2468] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:48:22.690495 27175 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:22.690750 28702 raft_consensus.cc:3060] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:22.625927 32541 raft_consensus.cc:493] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:22.693518 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.696707 27110 raft_consensus.cc:2468] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:22.698891 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.704519 27103 raft_consensus.cc:1275] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260807 01:48:22.707753 28709 raft_consensus.cc:1275] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260807 01:48:22.699033 32541 raft_consensus.cc:3060] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Advancing to term 5
I20260807 01:48:22.696909 32126 raft_consensus.cc:2804] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:48:22.698407 32546 raft_consensus.cc:2804] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 10 FOLLOWER]: Leader election won for term 10
W20260807 01:48:22.729861 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:22.736889 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:22.750609 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:22.755692 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.691730 32512 raft_consensus.cc:515] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.642076 32251 consensus_queue.cc:260] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:22.774660 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 79185.9us
I20260807 01:48:22.776549 28702 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 8.
I20260807 01:48:22.785583 27289 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:22.786755 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:22.703562 32482 raft_consensus.cc:515] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.789168 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:22.787467 27213 raft_consensus.cc:2376] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6. Re-sending same reply.
I20260807 01:48:22.787351 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf07cc968f3343869d12c10e81ce75f9" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:22.792172 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.792912 27212 raft_consensus.cc:2468] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
W20260807 01:48:22.777453 32513 log.cc:927] Time spent T 00000000000000000000000000000000 P 31e8851a2fee40aa9ea5af43f94890df: Append to log took a long time: real 0.080s user 0.001s sys 0.000s
I20260807 01:48:22.795492 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:48:22.796334 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 205546us
I20260807 01:48:22.776671 32546 raft_consensus.cc:697] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [term 10 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:22.801205 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:22.779918 32126 raft_consensus.cc:697] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [term 6 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:22.803349 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
W20260807 01:48:22.803789 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 208002us
I20260807 01:48:22.804728 27109 raft_consensus.cc:2376] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7. Re-sending same reply.
I20260807 01:48:22.801873 27103 raft_consensus.cc:2376] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 10. Re-sending same reply.
I20260807 01:48:22.762302 32492 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.815407 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.792111 32251 raft_consensus.cc:3060] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:22.808089 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b4fd523ba5d463b8508a943e293612c" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.819139 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.822939 27218 raft_consensus.cc:2368] T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for earlier term 5. Current term is 6.
I20260807 01:48:22.823185 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.825681 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1688e72d717b430e86793ecd17b2e952" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:22.826368 27104 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Advancing to term 10
I20260807 01:48:22.827369 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.828158 27100 raft_consensus.cc:2393] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 8: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:22.830008 28702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:22.830987 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.831929 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.832674 27213 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 8.
I20260807 01:48:22.833000 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:22.833617 27065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:22.833827 27100 raft_consensus.cc:2410] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260807 01:48:22.804497 32510 raft_consensus.cc:2804] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader election won for term 8
I20260807 01:48:22.839092 24592 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 01:48:22.842902 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18526f467154c49b830de9b1b03066f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:22.843582 27110 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Advancing to term 10
I20260807 01:48:22.795687 32139 consensus_queue.cc:1048] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260807 01:48:22.848960 27063 leader_election.cc:304] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:22.850142 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.819744 32110 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.852630 32110 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:22.853377 32510 raft_consensus.cc:697] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [term 8 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:22.857477 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.867887 27108 raft_consensus.cc:2468] T de3b20c9b4a7467ea9e6c098167df7e3 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:48:22.821625 31948 leader_election.cc:290] T b2dac5b83a404fbba1506670f8b8b45f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:22.873005 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.876981 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.877785 27213 raft_consensus.cc:2468] T 81439f7f101341049bd086b353ca3495 P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 4.
I20260807 01:48:22.855011 32510 consensus_queue.cc:237] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.884491 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.892350 27218 raft_consensus.cc:2368] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 7. Current term is 8.
I20260807 01:48:22.819301 32483 consensus_queue.cc:1059] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260807 01:48:22.898905 28710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
W20260807 01:48:22.898643 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 101268us
W20260807 01:48:22.905382 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:22.870328 32043 raft_consensus.cc:2749] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 01:48:22.843824 32493 consensus_queue.cc:1048] T 30561911ad3945159907e917cc1bafa7 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260807 01:48:22.921222 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:22.921873 27103 raft_consensus.cc:3060] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Advancing to term 10
I20260807 01:48:22.894095 32428 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:22.855428 31908 raft_consensus.cc:515] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.950424 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6420178dae2448f78216552b1b03c6e6" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:22.890655 32500 consensus_queue.cc:1048] T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260807 01:48:22.891049 32497 consensus_queue.cc:1048] T 67fed460b08e4124a6d4484db52478b1 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.030s
I20260807 01:48:22.953565 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.955300 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfefe1ff8f14fb68678ac128e129d32" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:22.914044 32547 raft_consensus.cc:493] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:22.918642 32549 raft_consensus.cc:3055] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 7 LEADER]: Stepping down as leader of term 7
I20260807 01:48:22.913591 32126 consensus_queue.cc:237] T 498dfe25861a4b4fbf2aca930b357274 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:22.964660 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 46310.3us
I20260807 01:48:22.898043 32546 consensus_queue.cc:237] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:22.919380 32551 consensus_queue.cc:1059] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260807 01:48:22.964917 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:22.983945 27226 raft_consensus.cc:2410] T 30561911ad3945159907e917cc1bafa7 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260807 01:48:22.985399 27100 raft_consensus.cc:1240] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [term 14 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 13. Current term is 14. Ops: []
I20260807 01:48:22.992798 28655 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:22.995232 27104 raft_consensus.cc:2468] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 10.
I20260807 01:48:22.996358 27110 raft_consensus.cc:2468] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 10.
W20260807 01:48:23.022274 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:22.983080 32551 raft_consensus.cc:3055] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 9 LEADER]: Stepping down as leader of term 9
I20260807 01:48:22.960382 32549 raft_consensus.cc:740] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 7 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
W20260807 01:48:23.047336 27287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:23.049558 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:23.050371 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
W20260807 01:48:23.116068 27287 connection.cc:256] RPC call timeout handler was delayed by -3.221756902989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19240000000000002s for a response
W20260807 01:48:23.116796 27287 connection.cc:256] RPC call timeout handler was delayed by -3.2172569029498845s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19290000000000002s for a response
W20260807 01:48:23.117117 27287 connection.cc:256] RPC call timeout handler was delayed by -3.1938569031190127s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1955s for a response
I20260807 01:48:23.056766 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b00b70a17a64e288e95507deb091fc1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
W20260807 01:48:23.117561 27287 connection.cc:256] RPC call timeout handler was delayed by -2.9823569031432271s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.219s for a response
I20260807 01:48:23.055773 28660 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
W20260807 01:48:23.120071 27287 connection.cc:256] RPC call timeout handler was delayed by -2.9814569030422717s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2191s for a response
W20260807 01:48:23.121327 27287 connection.cc:256] RPC call timeout handler was delayed by -2.952656903071329s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2223s for a response
W20260807 01:48:23.121728 27287 connection.cc:256] RPC call timeout handler was delayed by -2.6196569029707462s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25930000000000003s for a response
W20260807 01:48:23.122254 27287 connection.cc:256] RPC call timeout handler was delayed by -2.4324569031596184s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2801s for a response
W20260807 01:48:23.122637 27287 connection.cc:256] RPC call timeout handler was delayed by -2.3784569031558931s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2861s for a response
I20260807 01:48:23.124997 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:23.125753 27336 raft_consensus.cc:2468] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:48:23.127686 27336 raft_consensus.cc:1240] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260807 01:48:23.128912 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:23.129833 27336 raft_consensus.cc:2468] T 30561911ad3945159907e917cc1bafa7 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:48:23.121728 28655 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:23.137307 28655 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
W20260807 01:48:23.131428 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 75726.4us
I20260807 01:48:23.139472 28660 leader_election.cc:304] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
W20260807 01:48:23.174093 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:23.196610 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:23.134666 27064 leader_election.cc:304] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:22.984723 32547 raft_consensus.cc:515] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:23.207310 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3fc468354b48cf80eee526def18e6c" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:23.207855 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:23.208928 27330 raft_consensus.cc:2410] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260807 01:48:23.204391 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 67137.7us
I20260807 01:48:23.043612 28702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b18526f467154c49b830de9b1b03066f"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:34614
I20260807 01:48:23.211043 28702 raft_consensus.cc:606] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 9 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:48:23.212616 27336 raft_consensus.cc:3060] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Advancing to term 10
I20260807 01:48:23.213748 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:23.214068 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81439f7f101341049bd086b353ca3495" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:23.214535 27326 raft_consensus.cc:2468] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:23.214887 27331 raft_consensus.cc:2468] T 81439f7f101341049bd086b353ca3495 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:48:23.215240 27327 raft_consensus.cc:1240] T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260807 01:48:23.218024 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 83424.1us
I20260807 01:48:23.117882 32270 raft_consensus.cc:2804] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 01:48:23.221733 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:23.224093 27103 raft_consensus.cc:2468] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 10.
I20260807 01:48:23.219264 27064 leader_election.cc:304] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:23.226080 27338 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:23.223521 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:23.227082 27213 raft_consensus.cc:3060] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:23.215963 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:23.229283 27325 raft_consensus.cc:2393] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 6: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:23.039685 31908 leader_election.cc:290] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:23.048398 32551 raft_consensus.cc:740] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:23.230612 27064 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:23.228644 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:23.231998 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:23.233235 27328 raft_consensus.cc:1240] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Rejecting Update request from peer 3478729505b046c2a0051c2d0130873f for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260807 01:48:23.130357 32130 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:23.236003 32130 raft_consensus.cc:515] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:23.031459 32558 consensus_queue.cc:1059] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.186s
W20260807 01:48:23.236404 32575 log.cc:927] Time spent T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.191s user 0.000s sys 0.000s
I20260807 01:48:23.040199 32493 consensus_queue.cc:1048] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.183s
I20260807 01:48:23.238225 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:23.241541 32547 leader_election.cc:290] T b510849c3131475d8c1e032378dc893b P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:23.242250 32546 consensus_queue.cc:1048] T 60a03ab231464be89d6cce9e6988cfba P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 01:48:23.286872 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30561911ad3945159907e917cc1bafa7" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:23.242882 32318 raft_consensus.cc:2804] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:23.287629 32318 raft_consensus.cc:493] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:23.288033 32318 raft_consensus.cc:3060] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
W20260807 01:48:23.288504 32545 consensus_meta.cc:287] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 0.635s user 0.010s sys 0.000s
I20260807 01:48:23.288978 32545 raft_consensus.cc:515] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:23.243671 32549 consensus_queue.cc:260] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:23.289928 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:23.290786 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:23.288146 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:23.291443 27103 raft_consensus.cc:2468] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 6.
I20260807 01:48:23.036768 32571 consensus_queue.cc:1059] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.222s
I20260807 01:48:23.231264 32473 raft_consensus.cc:2804] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 01:48:23.209151 32529 raft_consensus.cc:2764] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260807 01:48:23.228679 32270 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:23.117506 32541 raft_consensus.cc:515] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:23.251062 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:23.233685 32500 raft_consensus.cc:2804] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:23.251763 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:23.258926 27338 raft_consensus.cc:2468] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8.
I20260807 01:48:23.304229 27331 raft_consensus.cc:2393] T 6b0f72aee1954e3ab89690c7cfb93677 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 7: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:23.239581 27064 leader_election.cc:304] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:23.263034 32565 raft_consensus.cc:2804] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 01:48:23.264705 32551 consensus_queue.cc:260] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:23.266391 32410 raft_consensus.cc:2764] T de3b20c9b4a7467ea9e6c098167df7e3 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260807 01:48:23.267015 32558 consensus_queue.cc:1048] T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 01:48:23.243863 27327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:23.271772 27336 raft_consensus.cc:2468] T 3b3fc468354b48cf80eee526def18e6c P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 10.
I20260807 01:48:23.272202 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "965af9378c6a4617b1d50f229ffc2c5b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:23.273530 27328 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:48:23.328495 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c4340b503246eca370314d2781034a" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:23.329282 27328 raft_consensus.cc:2468] T c4c4340b503246eca370314d2781034a P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 10.
I20260807 01:48:23.273109 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67fed460b08e4124a6d4484db52478b1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:23.330488 27330 raft_consensus.cc:2468] T 67fed460b08e4124a6d4484db52478b1 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:23.331431 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73e9ae0228aa41a996315efe07c04299" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:23.332159 27330 raft_consensus.cc:2468] T 73e9ae0228aa41a996315efe07c04299 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 4.
I20260807 01:48:23.333588 27330 raft_consensus.cc:1275] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 01:48:23.275843 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:23.276450 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:23.334743 27101 raft_consensus.cc:2468] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 9.
I20260807 01:48:23.336221 27101 raft_consensus.cc:1275] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260807 01:48:23.276535 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:23.337500 27102 raft_consensus.cc:2410] T fabde7ae28974f7380cb7641ab9fd6b1 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260807 01:48:23.276737 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:23.250936 32577 log.cc:927] Time spent T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.133s user 0.000s sys 0.000s
I20260807 01:48:23.286752 27110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:23.339210 27110 raft_consensus.cc:2468] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 5.
I20260807 01:48:23.289505 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:23.294544 27109 raft_consensus.cc:2393] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 8: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:23.295077 28710 raft_consensus.cc:2368] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 7. Current term is 8.
I20260807 01:48:23.295930 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:23.340122 27100 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 7.
I20260807 01:48:23.304872 27213 raft_consensus.cc:2468] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
W20260807 01:48:23.305109 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:23.305141 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 65636.9us
I20260807 01:48:23.341020 27337 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:48:23.307440 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:23.311355 27325 raft_consensus.cc:2410] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260807 01:48:23.315248 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1089e78f8246fcb8af73c74af99b19" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:23.314514 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:23.346096 27338 raft_consensus.cc:3060] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:23.344681 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3b20c9b4a7467ea9e6c098167df7e3" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:23.347047 27328 raft_consensus.cc:3060] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:23.340410 27336 raft_consensus.cc:1275] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260807 01:48:23.346288 27103 raft_consensus.cc:3060] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:23.349992 27218 raft_consensus.cc:2393] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 7: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:23.350633 27326 raft_consensus.cc:2468] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 5.
I20260807 01:48:23.295686 32524 raft_consensus.cc:2804] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260807 01:48:23.349886 32251 consensus_meta.cc:287] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 0.520s user 0.009s sys 0.001s
I20260807 01:48:23.310854 32043 raft_consensus.cc:2804] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 01:48:23.295879 32473 raft_consensus.cc:493] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:23.359128 32482 leader_election.cc:290] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:23.360662 32512 leader_election.cc:290] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:23.310706 32565 raft_consensus.cc:493] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:48:23.364137 32545 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:23.303599 32270 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:23.368454 27103 raft_consensus.cc:2468] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcf4e45defb94276bcfa68392711dade in term 8.
I20260807 01:48:23.359035 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:23.432698 27108 raft_consensus.cc:2393] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 7: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:23.343664 27064 leader_election.cc:304] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:23.310978 32500 raft_consensus.cc:493] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:23.356002 32551 raft_consensus.cc:3060] T 1688e72d717b430e86793ecd17b2e952 P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Advancing to term 10
W20260807 01:48:23.378909 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 166076us
I20260807 01:48:23.378943 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89dc6a602e9e42399d533f67d4bd9bef" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:23.382865 27337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:23.438648 27337 raft_consensus.cc:3060] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:23.439056 27067 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:23.418031 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:23.440229 27336 raft_consensus.cc:2468] T 05e95c9ad127429dbdd5252c0670f22f P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:48:23.429112 27330 raft_consensus.cc:1275] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260807 01:48:23.429639 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4098be3cca39416abf4b724092db65e4" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:48:23.414987 32591 log.cc:927] Time spent T 4098be3cca39416abf4b724092db65e4 P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.146s user 0.000s sys 0.005s
I20260807 01:48:23.435663 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:23.409942 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08edf043c99f445b99e723211e8e0ad0" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:23.443812 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:23.381139 32529 raft_consensus.cc:3055] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:48:23.444499 27325 raft_consensus.cc:2410] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260807 01:48:23.445060 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:23.444541 27331 raft_consensus.cc:2468] T 08edf043c99f445b99e723211e8e0ad0 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 9.
I20260807 01:48:23.416003 32473 raft_consensus.cc:3060] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Advancing to term 10
W20260807 01:48:23.447417 28662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:23.408548 32043 raft_consensus.cc:493] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:23.435833 32500 raft_consensus.cc:3060] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:23.454612 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:23.455364 27336 raft_consensus.cc:2468] T b510849c3131475d8c1e032378dc893b P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
I20260807 01:48:23.428333 32565 raft_consensus.cc:3060] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:23.462329 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8767bf1fc98c4effa8f3b5f5113afe40" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:23.466265 27331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a8b225f3fd4b65a1f62d70c6fd2bb7" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
W20260807 01:48:23.467033 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 342191us
W20260807 01:48:23.461058 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 321642us
I20260807 01:48:23.470134 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:23.470237 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:23.471081 27336 raft_consensus.cc:2468] T bf9ccfe7f7e64481b739c6efef03b95a P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 5.
I20260807 01:48:23.471190 27325 raft_consensus.cc:2393] T f27eab393b1f4f21aad32707749effa3 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 7: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:23.472527 27330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:23.473181 27330 raft_consensus.cc:3060] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Advancing to term 8
I20260807 01:48:23.473870 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:23.474815 27213 raft_consensus.cc:2393] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 8: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:23.477569 27338 raft_consensus.cc:2468] T ed1089e78f8246fcb8af73c74af99b19 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 9.
I20260807 01:48:23.468708 27287 leader_election.cc:304] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:23.480865 27172 leader_election.cc:304] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:23.482108 27172 leader_election.cc:304] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:23.456280 32043 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:23.447949 32529 raft_consensus.cc:740] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:23.483779 27172 leader_election.cc:304] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:23.482932 27218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:23.487169 27218 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Advancing to term 10
I20260807 01:48:23.486546 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3806d33df8144ca5a662db5461461b43" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:23.488592 27172 leader_election.cc:304] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:23.498828 27172 leader_election.cc:304] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:23.483513 32493 raft_consensus.cc:2804] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:23.497071 27338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
W20260807 01:48:23.501834 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 51419.8us
I20260807 01:48:23.503041 32425 raft_consensus.cc:2804] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:23.503535 32425 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:23.503888 32425 raft_consensus.cc:3060] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:23.504220 27172 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12
W20260807 01:48:23.500414 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 156814us
I20260807 01:48:23.506978 27336 raft_consensus.cc:1275] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 13 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260807 01:48:23.487226 32614 consensus_queue.cc:1048] T fabde7ae28974f7380cb7641ab9fd6b1 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260807 01:48:23.505388 32126 raft_consensus.cc:2804] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader election won for term 8
I20260807 01:48:23.506315 27064 leader_election.cc:304] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:23.493078 32547 raft_consensus.cc:2804] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260807 01:48:23.534087 27170 leader_election.cc:336] T 67fed460b08e4124a6d4484db52478b1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.569s (SENT)
I20260807 01:48:23.514922 27287 leader_election.cc:304] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:23.541231 27337 raft_consensus.cc:2468] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8.
I20260807 01:48:23.541980 28660 leader_election.cc:304] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3
I20260807 01:48:23.514365 32130 leader_election.cc:290] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:23.488785 32363 raft_consensus.cc:2804] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader election won for term 9
I20260807 01:48:23.486691 32610 raft_consensus.cc:2804] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Leader election won for term 9
I20260807 01:48:23.544111 27330 raft_consensus.cc:2468] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 8.
W20260807 01:48:23.550645 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 48123.9us
I20260807 01:48:23.543238 28660 leader_election.cc:304] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:23.545022 32363 raft_consensus.cc:697] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [term 9 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:23.546130 32594 raft_consensus.cc:2804] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:48:23.514775 32126 raft_consensus.cc:697] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [term 8 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:23.486830 32510 raft_consensus.cc:2749] T d235fc7e09b8465f8ddc570370605464 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:23.550531 32511 raft_consensus.cc:2804] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader election won for term 5
W20260807 01:48:23.552536 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 113550us
I20260807 01:48:23.501583 32493 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:23.501346 32541 leader_election.cc:290] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:23.510102 32614 raft_consensus.cc:2764] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260807 01:48:23.541090 32547 raft_consensus.cc:493] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:23.572054 32547 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:23.574503 32126 consensus_queue.cc:237] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:23.575366 27287 leader_election.cc:304] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:23.574484 32251 raft_consensus.cc:2804] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:23.576197 32251 raft_consensus.cc:493] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:23.576601 32251 raft_consensus.cc:3060] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:23.578404 32139 raft_consensus.cc:2749] T 51cef821f0284d8e9369e98d38f80e73 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:23.579439 32139 consensus_queue.cc:1048] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 01:48:23.558421 32610 raft_consensus.cc:697] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [term 9 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:23.582760 32610 consensus_queue.cc:237] T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:23.574379 32511 raft_consensus.cc:697] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [term 5 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:23.584736 32511 consensus_queue.cc:237] T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:23.559350 28660 leader_election.cc:304] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:23.566074 27067 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:23.562481 32493 raft_consensus.cc:3060] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Advancing to term 6
W20260807 01:48:23.604831 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:23.608871 32492 leader_election.cc:290] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:23.611243 32043 raft_consensus.cc:515] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:23.612628 27105 raft_consensus.cc:2368] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 8. Current term is 9.
I20260807 01:48:23.613293 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498dfe25861a4b4fbf2aca930b357274" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:23.614082 28714 raft_consensus.cc:2368] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 5. Current term is 6.
W20260807 01:48:23.583707 32593 log.cc:927] Time spent T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.090s user 0.000s sys 0.000s
I20260807 01:48:23.570794 32363 consensus_queue.cc:237] T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:23.593768 27287 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
W20260807 01:48:23.594636 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:23.596393 27328 raft_consensus.cc:2468] T de3b20c9b4a7467ea9e6c098167df7e3 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:48:23.596808 27218 raft_consensus.cc:2468] T 270fd0241ee84fcea8889fff6d973747 P 3478729505b046c2a0051c2d0130873f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 10.
I20260807 01:48:23.600785 28702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
mode: GRACEFUL
new_leader_uuid: "fcf4e45defb94276bcfa68392711dade"
from {username='slave'} at 127.0.0.1:34614
W20260807 01:48:23.619563 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 53658.8us
I20260807 01:48:23.634931 27067 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:23.622148 32562 raft_consensus.cc:2749] T 67fed460b08e4124a6d4484db52478b1 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:48:23.623160 27289 leader_election.cc:400] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 election: Vote denied by peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225) with higher term. Message: Invalid argument: T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 5. Current term is 6.
I20260807 01:48:23.632877 28702 raft_consensus.cc:606] T b2dac5b83a404fbba1506670f8b8b45f P 57bb4985157447f0a3ab680fa0026f12 [term 4 LEADER]: Received request to transfer leadership to TS fcf4e45defb94276bcfa68392711dade
I20260807 01:48:23.635792 27289 leader_election.cc:403] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260807 01:48:23.623786 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 101228us
I20260807 01:48:23.628077 32425 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:23.629195 32574 raft_consensus.cc:3055] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 9 LEADER]: Stepping down as leader of term 9
W20260807 01:48:23.620972 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 61683.3us
I20260807 01:48:23.638209 27287 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:23.611435 32624 raft_consensus.cc:2804] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 01:48:23.644676 28660 leader_election.cc:304] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:23.593271 32594 raft_consensus.cc:697] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [term 6 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
W20260807 01:48:23.645569 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 141398us
I20260807 01:48:23.647867 28660 leader_election.cc:400] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for earlier term 5. Current term is 6.
I20260807 01:48:23.648916 27287 leader_election.cc:304] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:23.632942 32043 leader_election.cc:290] T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:23.640877 32541 raft_consensus.cc:2804] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 01:48:23.693563 28660 leader_election.cc:403] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260807 01:48:23.697834 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 50133.5us
I20260807 01:48:23.643529 32574 raft_consensus.cc:740] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:23.644739 32622 raft_consensus.cc:2749] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:48:23.691042 32482 raft_consensus.cc:2804] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 01:48:23.691340 32562 raft_consensus.cc:2749] T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:23.699644 32043 raft_consensus.cc:2804] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 01:48:23.694095 32428 raft_consensus.cc:2749] T 9f9f47b3d3124bda8fe235e759b1d061 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 01:48:23.646059 31948 raft_consensus.cc:3060] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:23.691151 27172 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:23.704273 28660 leader_election.cc:304] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:23.695119 32541 raft_consensus.cc:493] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:23.713625 27172 leader_election.cc:304] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:23.648324 32318 raft_consensus.cc:515] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:23.646739 32500 raft_consensus.cc:515] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:23.721026 27172 leader_election.cc:304] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:23.698412 32628 raft_consensus.cc:2749] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 01:48:23.712580 32595 raft_consensus.cc:2804] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader election won for term 7
W20260807 01:48:23.708302 27067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 73422.8us
I20260807 01:48:23.704468 32555 raft_consensus.cc:2749] T 2b4fd523ba5d463b8508a943e293612c P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855) with higher term. Message: Invalid argument: T 2b4fd523ba5d463b8508a943e293612c P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 for earlier term 5. Current term is 6.
I20260807 01:48:23.724406 27172 leader_election.cc:304] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:23.729327 28702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3ee061f38f84a71ad3991835a2e56dc"
dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
mode: GRACEFUL
new_leader_uuid: "3262a9d183914b699227fdba0ca5bed3"
from {username='slave'} at 127.0.0.1:34614
I20260807 01:48:23.730142 28702 raft_consensus.cc:606] T b3ee061f38f84a71ad3991835a2e56dc P 57bb4985157447f0a3ab680fa0026f12 [term 8 LEADER]: Received request to transfer leadership to TS 3262a9d183914b699227fdba0ca5bed3
I20260807 01:48:23.730839 32524 raft_consensus.cc:493] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting leader election (detected failure of leader fcf4e45defb94276bcfa68392711dade)
I20260807 01:48:23.731364 32524 raft_consensus.cc:3060] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:23.727979 28660 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:23.708691 32482 raft_consensus.cc:493] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3262a9d183914b699227fdba0ca5bed3)
I20260807 01:48:23.710575 32043 raft_consensus.cc:493] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:23.698977 32270 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:23.737512 27172 leader_election.cc:304] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:23.739777 32545 raft_consensus.cc:2749] T 8767bf1fc98c4effa8f3b5f5113afe40 P 57bb4985157447f0a3ab680fa0026f12 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260807 01:48:23.740334 32595 raft_consensus.cc:697] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [term 7 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:23.721285 32541 raft_consensus.cc:3060] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:23.724668 32614 raft_consensus.cc:2804] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election won for term 7
W20260807 01:48:23.752247 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:23.732903 32500 leader_election.cc:290] T 9db18f8163fc4d79aceac26f0e762e2b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:23.752182 27067 leader_election.cc:304] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters:
I20260807 01:48:23.733469 31908 raft_consensus.cc:2804] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 10 FOLLOWER]: Leader election won for term 10
I20260807 01:48:23.720209 32627 raft_consensus.cc:2749] T b3ee061f38f84a71ad3991835a2e56dc P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 01:48:23.722643 32492 consensus_queue.cc:1048] T 21e1a5771ca04812aac6f5abed966ccd P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260807 01:48:23.753520 32126 raft_consensus.cc:2749] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 01:48:23.738884 32482 raft_consensus.cc:3060] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:23.746740 32043 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:23.761538 32546 consensus_queue.cc:1059] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
W20260807 01:48:23.769613 32629 log.cc:927] Time spent T 9db18f8163fc4d79aceac26f0e762e2b P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.060s user 0.001s sys 0.000s
I20260807 01:48:23.724812 32473 raft_consensus.cc:515] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:23.753324 32610 raft_consensus.cc:2804] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:48:23.767479 32614 raft_consensus.cc:697] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [term 7 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:23.777179 32637 raft_consensus.cc:2764] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260807 01:48:23.778026 32546 consensus_queue.cc:1243] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [LEADER]: Peer 3262a9d183914b699227fdba0ca5bed3 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260807 01:48:23.799547 32473 leader_election.cc:290] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:23.800149 32565 raft_consensus.cc:515] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:23.801442 28709 raft_consensus.cc:1240] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [term 4 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 3. Current term is 4. Ops: []
I20260807 01:48:23.769650 32493 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:23.808845 32493 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:23.809549 32428 consensus_queue.cc:1059] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 01:48:23.811088 32428 raft_consensus.cc:3055] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Stepping down as leader of term 3
I20260807 01:48:23.811391 32270 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:48:23.812182 32598 log.cc:927] Time spent T 8b00b70a17a64e288e95507deb091fc1 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.304s user 0.002s sys 0.000s
I20260807 01:48:23.812712 32614 consensus_queue.cc:237] T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:23.813783 28709 raft_consensus.cc:1275] T 2c3fdef784cc4afbb8dda87287a7eac0 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Refusing update from remote peer fcf4e45defb94276bcfa68392711dade: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260807 01:48:23.775065 31908 raft_consensus.cc:697] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [term 10 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:23.814899 31908 consensus_queue.cc:237] T 3b3fc468354b48cf80eee526def18e6c P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:23.816262 32139 consensus_queue.cc:1048] T 2c3fdef784cc4afbb8dda87287a7eac0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 01:48:23.817265 32482 raft_consensus.cc:515] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:23.821050 27325 raft_consensus.cc:1240] T 89dc6a602e9e42399d533f67d4bd9bef P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 5. Current term is 6. Ops: []
I20260807 01:48:23.774075 32624 raft_consensus.cc:697] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [term 6 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:23.789160 32610 raft_consensus.cc:697] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [term 7 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:23.789623 32595 consensus_queue.cc:237] T f7a8b225f3fd4b65a1f62d70c6fd2bb7 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:23.767354 32594 consensus_queue.cc:237] T cf4d8c8825e149adbd4565e0c8fdd109 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:23.791741 32318 leader_election.cc:290] T cf4d8c8825e149adbd4565e0c8fdd109 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: Requested vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:23.783412 32425 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:23.826820 27328 raft_consensus.cc:1240] T 2cfefe1ff8f14fb68678ac128e129d32 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 5. Current term is 6. Ops: []
I20260807 01:48:23.830447 27325 raft_consensus.cc:1240] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 9. Current term is 10. Ops: []
I20260807 01:48:23.835027 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:23.836529 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:23.837898 28655 leader_election.cc:304] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
W20260807 01:48:23.851244 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 160042us
I20260807 01:48:23.852772 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:23.864835 27336 raft_consensus.cc:3060] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Advancing to term 9
I20260807 01:48:23.859117 28715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:23.793826 32547 raft_consensus.cc:515] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:23.811522 32428 raft_consensus.cc:740] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:23.897275 27289 leader_election.cc:400] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 election: Vote denied by peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225) with higher term. Message: Invalid argument: T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 7. Current term is 8.
I20260807 01:48:23.897818 27289 leader_election.cc:403] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260807 01:48:23.812392 32251 raft_consensus.cc:515] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:23.903465 27064 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 260686us
I20260807 01:48:23.904412 27064 leader_election.cc:304] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
W20260807 01:48:23.919413 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 81561.7us
I20260807 01:48:23.921303 27064 leader_election.cc:304] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, 57bb4985157447f0a3ab680fa0026f12
W20260807 01:48:23.922811 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 57041.4us
I20260807 01:48:23.921206 28655 leader_election.cc:304] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:23.901721 32565 leader_election.cc:290] T a0e280c976844349b4874a479aa08de2 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 4 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:23.908865 31948 raft_consensus.cc:2749] T 498dfe25861a4b4fbf2aca930b357274 P fcf4e45defb94276bcfa68392711dade [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225) with higher term. Message: Invalid argument: T 498dfe25861a4b4fbf2aca930b357274 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 5. Current term is 6.
W20260807 01:48:23.960137 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:23.955299 27064 leader_election.cc:304] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:23.962015 28715 raft_consensus.cc:2393] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 6: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
W20260807 01:48:23.962297 27175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:23.964475 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a03ab231464be89d6cce9e6988cfba" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:23.965236 27104 raft_consensus.cc:2468] T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 6.
I20260807 01:48:23.924793 32545 raft_consensus.cc:2804] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:23.922729 32318 raft_consensus.cc:2749] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 01:48:23.967413 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:23.968544 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f27eab393b1f4f21aad32707749effa3" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:23.993220 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:23.993849 27104 raft_consensus.cc:3060] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:23.969359 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:23.970306 27101 raft_consensus.cc:1240] T 1688e72d717b430e86793ecd17b2e952 P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 9. Current term is 10. Ops: []
I20260807 01:48:23.995760 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:23.996691 27101 raft_consensus.cc:2393] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 6: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:23.964054 32428 consensus_queue.cc:260] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:23.971148 32644 log.cc:927] Time spent T 05e95c9ad127429dbdd5252c0670f22f P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.051s user 0.000s sys 0.001s
I20260807 01:48:23.971462 31948 consensus_queue.cc:1048] T 54eb75662df043488517f09bff1dfcff P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 01:48:23.956110 32473 raft_consensus.cc:2749] T b3ee061f38f84a71ad3991835a2e56dc P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 01:48:23.972512 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09799289505a4c188931e0f3ed0adfb3" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:23.971382 27212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:24.030894 27230 raft_consensus.cc:1240] T 4098be3cca39416abf4b724092db65e4 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260807 01:48:23.974498 27336 raft_consensus.cc:2468] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 9.
I20260807 01:48:23.976593 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21e1a5771ca04812aac6f5abed966ccd" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:23.979898 28707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:24.032698 27213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9db18f8163fc4d79aceac26f0e762e2b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:23.980784 32130 raft_consensus.cc:3055] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 13 LEADER]: Stepping down as leader of term 13
I20260807 01:48:23.962934 32538 consensus_queue.cc:1059] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260807 01:48:23.981478 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b510849c3131475d8c1e032378dc893b" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12" is_pre_election: true
I20260807 01:48:23.982316 27103 raft_consensus.cc:1240] T 565d0a4463ac45e4bad1886cc0095ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 9. Current term is 10. Ops: []
I20260807 01:48:23.982887 28708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:23.983167 27100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3fdef784cc4afbb8dda87287a7eac0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:23.983363 32646 log.cc:927] Time spent T ed1089e78f8246fcb8af73c74af99b19 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.075s user 0.001s sys 0.000s
I20260807 01:48:23.983911 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf9ccfe7f7e64481b739c6efef03b95a" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:23.984529 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:24.048363 27109 raft_consensus.cc:2468] T 54eb75662df043488517f09bff1dfcff P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 10.
I20260807 01:48:23.986372 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:23.987087 32635 log.cc:927] Time spent T 879666f362ba4a2db3fdfe6738b37dec P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.120s user 0.000s sys 0.001s
I20260807 01:48:23.986879 32610 consensus_queue.cc:237] T de3b20c9b4a7467ea9e6c098167df7e3 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } attrs { replace: true } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:24.052280 27103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "270fd0241ee84fcea8889fff6d973747" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:24.053267 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:23.989033 27227 raft_consensus.cc:1275] T 6420178dae2448f78216552b1b03c6e6 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260807 01:48:24.053663 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40f160d4dff84d8d9fb21b3489d01ff7" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:24.003984 27216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565d0a4463ac45e4bad1886cc0095ff7" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:24.021555 27102 raft_consensus.cc:1240] T b18526f467154c49b830de9b1b03066f P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 9. Current term is 10. Ops: []
I20260807 01:48:24.021605 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94962ef95ca14973bc39a17ed76c0087" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:24.054950 27108 raft_consensus.cc:2376] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 8. Re-sending same reply.
I20260807 01:48:23.917093 32624 consensus_queue.cc:237] T 2b4fd523ba5d463b8508a943e293612c P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:24.062482 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:23.975203 32500 raft_consensus.cc:2804] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Leader election won for term 10
I20260807 01:48:24.026944 27110 raft_consensus.cc:1240] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 8. Current term is 9. Ops: [6.7-8.8]
I20260807 01:48:24.026661 27099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f9f47b3d3124bda8fe235e759b1d061" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
W20260807 01:48:24.028105 27170 leader_election.cc:336] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 2.201s (SENT)
I20260807 01:48:24.071030 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:24.071683 27170 leader_election.cc:304] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:24.028867 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:24.033409 27213 raft_consensus.cc:3060] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:24.053155 32571 raft_consensus.cc:3055] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 01:48:24.073576 27101 raft_consensus.cc:3060] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:24.032698 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e18e65b8d2418cb159040f0fae8be3" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:24.070997 27099 raft_consensus.cc:2468] T 9f9f47b3d3124bda8fe235e759b1d061 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 6.
W20260807 01:48:24.076510 32647 log.cc:927] Time spent T bf07cc968f3343869d12c10e81ce75f9 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.118s user 0.001s sys 0.000s
I20260807 01:48:24.077220 27099 raft_consensus.cc:1240] T 8b00b70a17a64e288e95507deb091fc1 P 3262a9d183914b699227fdba0ca5bed3 [term 5 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 4. Current term is 5. Ops: []
I20260807 01:48:24.034399 27212 raft_consensus.cc:2393] T d8e18e65b8d2418cb159040f0fae8be3 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade in current term 8: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:24.079730 32574 consensus_queue.cc:260] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:24.080982 32574 raft_consensus.cc:3060] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Advancing to term 10
I20260807 01:48:24.034951 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0e280c976844349b4874a479aa08de2" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:23.965637 32627 raft_consensus.cc:493] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:23.990833 32545 raft_consensus.cc:493] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:23.974071 32483 raft_consensus.cc:2749] T d53e096d3ad04a5bb333660000313179 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225) with higher term. Message: Invalid argument: T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 7. Current term is 8.
I20260807 01:48:24.049546 27105 raft_consensus.cc:2393] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 6: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:24.050236 27100 raft_consensus.cc:2393] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 8: Already voted for candidate fcf4e45defb94276bcfa68392711dade in this term.
I20260807 01:48:24.035696 32130 raft_consensus.cc:740] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:23.998764 28660 leader_election.cc:304] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12; no voters: 3478729505b046c2a0051c2d0130873f
I20260807 01:48:24.058041 28714 raft_consensus.cc:2393] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3478729505b046c2a0051c2d0130873f in current term 7: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
W20260807 01:48:24.091046 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 92366.1us
I20260807 01:48:24.023113 27214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf4d8c8825e149adbd4565e0c8fdd109" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:24.092675 27214 raft_consensus.cc:3060] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:24.065367 27227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dac5b83a404fbba1506670f8b8b45f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:24.075551 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e95c9ad127429dbdd5252c0670f22f" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:24.069697 32500 raft_consensus.cc:697] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [term 10 LEADER]: Becoming Leader. State: Replica: 3262a9d183914b699227fdba0ca5bed3, State: Running, Role: LEADER
I20260807 01:48:24.084438 27230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53e096d3ad04a5bb333660000313179" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:24.087365 32547 leader_election.cc:290] T b3a9345c5a2946019985c5a10aacd64f P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 9 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:24.050541 32043 raft_consensus.cc:515] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:24.057792 32428 raft_consensus.cc:3060] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 3 FOLLOWER]: Advancing to term 4
I20260807 01:48:24.046404 32524 raft_consensus.cc:515] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:24.052515 32251 leader_election.cc:290] T 9febf660f0fd492dabb2b64043136fbe P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 6 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:24.094015 27229 raft_consensus.cc:2410] T 05e95c9ad127429dbdd5252c0670f22f P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260807 01:48:24.120306 27230 raft_consensus.cc:2368] T d53e096d3ad04a5bb333660000313179 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 7. Current term is 8.
W20260807 01:48:24.168850 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 214269us
I20260807 01:48:24.171995 27226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:24.172852 27226 raft_consensus.cc:2393] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 6: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:24.093031 32545 raft_consensus.cc:3060] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12 [term 5 FOLLOWER]: Advancing to term 6
W20260807 01:48:24.100755 27170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 72683.3us
I20260807 01:48:24.137393 27104 raft_consensus.cc:2468] T 94962ef95ca14973bc39a17ed76c0087 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
I20260807 01:48:24.082996 32571 raft_consensus.cc:740] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:24.177022 27214 raft_consensus.cc:2468] T cf4d8c8825e149adbd4565e0c8fdd109 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 7.
I20260807 01:48:24.180850 28709 raft_consensus.cc:1240] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Rejecting Update request from peer fcf4e45defb94276bcfa68392711dade for earlier term 6. Current term is 7. Ops: []
I20260807 01:48:24.181035 27109 raft_consensus.cc:2393] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 9: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:24.091148 32482 leader_election.cc:290] T 270fd0241ee84fcea8889fff6d973747 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:24.131369 32555 raft_consensus.cc:2804] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:24.177840 27213 raft_consensus.cc:2468] T 9db18f8163fc4d79aceac26f0e762e2b P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 6.
I20260807 01:48:24.126960 32425 raft_consensus.cc:2749] T b2dac5b83a404fbba1506670f8b8b45f P 3478729505b046c2a0051c2d0130873f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:24.189874 32596 raft_consensus.cc:2804] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Leader election won for term 8
W20260807 01:48:24.183681 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.4:46225, user_credentials={real_user=slave}} blocked reactor thread for 286558us
I20260807 01:48:24.124325 32500 consensus_queue.cc:237] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:24.231315 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:24.190654 32596 raft_consensus.cc:697] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [term 8 LEADER]: Becoming Leader. State: Replica: 57bb4985157447f0a3ab680fa0026f12, State: Running, Role: LEADER
I20260807 01:48:24.242673 27101 raft_consensus.cc:2468] T a0e280c976844349b4874a479aa08de2 P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
I20260807 01:48:24.188273 32555 raft_consensus.cc:493] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
W20260807 01:48:24.171147 27063 leader_election.cc:336] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.548s (SENT)
W20260807 01:48:24.274664 28661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:24.278900 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 107775us
W20260807 01:48:24.279811 27063 leader_election.cc:336] T 270fd0241ee84fcea8889fff6d973747 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.614s (SENT)
I20260807 01:48:24.237442 32483 raft_consensus.cc:493] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:24.286796 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:24.092883 32627 raft_consensus.cc:515] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
W20260807 01:48:24.292418 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 368778us
I20260807 01:48:24.307823 27287 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:24.174351 32130 consensus_queue.cc:260] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:24.312393 27172 leader_election.cc:304] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3478729505b046c2a0051c2d0130873f; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:24.286595 32555 raft_consensus.cc:3060] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:24.306313 32651 raft_consensus.cc:493] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:24.334475 32651 raft_consensus.cc:515] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:24.336441 28714 raft_consensus.cc:1275] T 6420178dae2448f78216552b1b03c6e6 P 57bb4985157447f0a3ab680fa0026f12 [term 8 FOLLOWER]: Refusing update from remote peer 3262a9d183914b699227fdba0ca5bed3: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260807 01:48:24.337298 32651 leader_election.cc:290] T 6b0f72aee1954e3ab89690c7cfb93677 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:24.338318 32130 raft_consensus.cc:3060] T 3806d33df8144ca5a662db5461461b43 P 3478729505b046c2a0051c2d0130873f [term 13 FOLLOWER]: Advancing to term 14
I20260807 01:48:24.339251 32043 leader_election.cc:290] T 1c4e145f8f52431fa41b1f596e2f1133 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 election: Requested vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:24.339955 32043 consensus_queue.cc:1048] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260807 01:48:24.340315 28714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4e145f8f52431fa41b1f596e2f1133" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:24.341135 28714 raft_consensus.cc:2393] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 9: Already voted for candidate 57bb4985157447f0a3ab680fa0026f12 in this term.
W20260807 01:48:24.341660 27063 leader_election.cc:336] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.705s (SENT)
I20260807 01:48:24.342177 27063 leader_election.cc:304] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:24.342941 32524 leader_election.cc:290] T c8799906440c4ef19cfbadc18cf32ef0 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
W20260807 01:48:24.343736 27063 leader_election.cc:336] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.727s (SENT)
I20260807 01:48:24.345819 27222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f"
I20260807 01:48:24.347147 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade"
I20260807 01:48:24.332264 27172 leader_election.cc:304] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:24.344206 27063 leader_election.cc:304] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:24.351724 27101 raft_consensus.cc:1275] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260807 01:48:24.352349 27172 leader_election.cc:304] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:24.353639 32482 raft_consensus.cc:2749] T 37478d818c864abc83662e98299718d1 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260807 01:48:24.353796 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 46094.4us
I20260807 01:48:24.354532 32425 raft_consensus.cc:2804] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 8 FOLLOWER]: Leader election won for term 8
I20260807 01:48:24.354817 27287 leader_election.cc:304] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:24.360628 27101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3a9345c5a2946019985c5a10aacd64f" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:24.353492 32493 raft_consensus.cc:2749] T 2c3fdef784cc4afbb8dda87287a7eac0 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260807 01:48:24.361332 27101 raft_consensus.cc:3060] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Advancing to term 9
W20260807 01:48:24.360236 27063 leader_election.cc:336] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.877s (SENT)
I20260807 01:48:24.334365 32318 raft_consensus.cc:493] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:24.365391 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9febf660f0fd492dabb2b64043136fbe" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:24.373725 27104 raft_consensus.cc:2393] T 9febf660f0fd492dabb2b64043136fbe P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 6: Already voted for candidate 3262a9d183914b699227fdba0ca5bed3 in this term.
I20260807 01:48:24.362485 27063 leader_election.cc:304] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:24.300027 32483 raft_consensus.cc:515] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:24.346167 31908 raft_consensus.cc:2749] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 01:48:24.394134 27287 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
W20260807 01:48:24.355217 32677 log.cc:927] Time spent T bf07cc968f3343869d12c10e81ce75f9 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.072s user 0.000s sys 0.001s
I20260807 01:48:24.397459 27287 leader_election.cc:304] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
W20260807 01:48:24.394542 27063 leader_election.cc:336] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.879s (SENT)
I20260807 01:48:24.399065 27063 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 3478729505b046c2a0051c2d0130873f, fcf4e45defb94276bcfa68392711dade
I20260807 01:48:24.394974 27222 raft_consensus.cc:2393] T 9febf660f0fd492dabb2b64043136fbe P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 57bb4985157447f0a3ab680fa0026f12 in current term 6: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:24.380671 27229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b0f72aee1954e3ab89690c7cfb93677" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3478729505b046c2a0051c2d0130873f" is_pre_election: true
I20260807 01:48:24.385593 27101 raft_consensus.cc:2468] T b3a9345c5a2946019985c5a10aacd64f P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 9.
I20260807 01:48:24.387686 32483 leader_election.cc:290] T 4098be3cca39416abf4b724092db65e4 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855)
I20260807 01:48:24.355255 32043 raft_consensus.cc:493] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3478729505b046c2a0051c2d0130873f)
I20260807 01:48:24.398927 32493 consensus_queue.cc:1048] T 6420178dae2448f78216552b1b03c6e6 P 3262a9d183914b699227fdba0ca5bed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260807 01:48:24.422470 32043 raft_consensus.cc:515] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:24.423405 32673 raft_consensus.cc:2749] T 3806d33df8144ca5a662db5461461b43 P fcf4e45defb94276bcfa68392711dade [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260807 01:48:24.298888 32571 consensus_queue.cc:260] T 89dc6a602e9e42399d533f67d4bd9bef P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:24.425040 32043 leader_election.cc:290] T d235fc7e09b8465f8ddc570370605464 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:24.426352 32482 raft_consensus.cc:2804] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Leader election won for term 10
I20260807 01:48:24.426991 32482 raft_consensus.cc:697] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [term 10 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:24.395361 27172 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f; no voters:
I20260807 01:48:24.403091 32458 raft_consensus.cc:2804] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 01:48:24.428375 32458 raft_consensus.cc:493] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 01:48:24.428781 32458 raft_consensus.cc:3060] T 9f9f47b3d3124bda8fe235e759b1d061 P 3478729505b046c2a0051c2d0130873f [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:24.429524 32482 consensus_queue.cc:237] T 1688e72d717b430e86793ecd17b2e952 P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:24.361847 32425 raft_consensus.cc:697] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [term 8 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
I20260807 01:48:24.434942 32425 consensus_queue.cc:237] T 40f160d4dff84d8d9fb21b3489d01ff7 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:24.438671 31908 raft_consensus.cc:2804] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 7 FOLLOWER]: Leader election won for term 7
I20260807 01:48:24.448086 32664 consensus_queue.cc:1048] T 8767bf1fc98c4effa8f3b5f5113afe40 P 3478729505b046c2a0051c2d0130873f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260807 01:48:24.453053 32541 consensus_meta.cc:287] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade: Time spent flushing consensus metadata: real 0.690s user 0.010s sys 0.000s
I20260807 01:48:24.415530 27287 leader_election.cc:304] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:24.455493 32627 leader_election.cc:290] T 37478d818c864abc83662e98299718d1 P 3478729505b046c2a0051c2d0130873f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429)
I20260807 01:48:24.455906 32483 raft_consensus.cc:2804] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 10 FOLLOWER]: Leader election won for term 10
I20260807 01:48:24.456575 32483 raft_consensus.cc:697] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [term 10 LEADER]: Becoming Leader. State: Replica: fcf4e45defb94276bcfa68392711dade, State: Running, Role: LEADER
I20260807 01:48:24.457515 32483 consensus_queue.cc:237] T b18526f467154c49b830de9b1b03066f P fcf4e45defb94276bcfa68392711dade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:24.371801 32596 consensus_queue.cc:237] T d53e096d3ad04a5bb333660000313179 P 57bb4985157447f0a3ab680fa0026f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:24.423009 32663 log.cc:927] Time spent T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade: Append to log took a long time: real 0.178s user 0.001s sys 0.000s
I20260807 01:48:24.470940 27287 leader_election.cc:304] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:24.453279 27326 raft_consensus.cc:2468] T a0e280c976844349b4874a479aa08de2 P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 4.
W20260807 01:48:24.475544 27063 leader_election.cc:336] T 3806d33df8144ca5a662db5461461b43 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.937s (SENT)
W20260807 01:48:24.519783 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 44289.8us
W20260807 01:48:24.520495 27063 leader_election.cc:336] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.943s (SENT)
I20260807 01:48:24.520864 27063 leader_election.cc:304] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
W20260807 01:48:24.546285 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.2:34855, user_credentials={real_user=slave}} blocked reactor thread for 233586us
I20260807 01:48:24.362828 32637 raft_consensus.cc:2749] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260807 01:48:24.551040 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 80169.5us
I20260807 01:48:24.571076 27287 leader_election.cc:304] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade; no voters: 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:24.405113 32651 raft_consensus.cc:2749] T 00395257beb442e892a4ec18c4c30d95 P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260807 01:48:24.617357 27063 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 96888.9us
I20260807 01:48:24.549923 32662 raft_consensus.cc:2749] T 1c4e145f8f52431fa41b1f596e2f1133 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260807 01:48:24.639909 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 68913.3us
I20260807 01:48:24.661845 27287 leader_election.cc:304] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcf4e45defb94276bcfa68392711dade; no voters: 3262a9d183914b699227fdba0ca5bed3, 57bb4985157447f0a3ab680fa0026f12
I20260807 01:48:24.453629 32541 raft_consensus.cc:515] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } attrs { replace: true } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:24.393920 32318 raft_consensus.cc:515] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
I20260807 01:48:24.672649 28660 leader_election.cc:304] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, fcf4e45defb94276bcfa68392711dade
W20260807 01:48:24.655354 32614 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s user 0.034s sys 0.028s
I20260807 01:48:24.663410 32428 raft_consensus.cc:2804] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 01:48:24.629281 32511 raft_consensus.cc:2804] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 01:48:24.694653 27109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8799906440c4ef19cfbadc18cf32ef0" candidate_uuid: "57bb4985157447f0a3ab680fa0026f12" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3"
I20260807 01:48:24.676462 32043 raft_consensus.cc:493] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:24.720042 27172 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 291290us
W20260807 01:48:24.689257 32614 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.610s user 0.035s sys 0.028s
I20260807 01:48:24.714653 27333 raft_consensus.cc:1240] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Rejecting Update request from peer 57bb4985157447f0a3ab680fa0026f12 for earlier term 8. Current term is 9. Ops: []
I20260807 01:48:24.629009 32679 raft_consensus.cc:2749] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 01:48:24.669450 32648 consensus_queue.cc:1048] T c8799906440c4ef19cfbadc18cf32ef0 P fcf4e45defb94276bcfa68392711dade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.119s
I20260807 01:48:24.702461 32662 raft_consensus.cc:2749] T 965af9378c6a4617b1d50f229ffc2c5b P fcf4e45defb94276bcfa68392711dade [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 01:48:24.726420 32511 raft_consensus.cc:493] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 01:48:24.777594 27288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:24.779608 27109 raft_consensus.cc:3060] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 6 FOLLOWER]: Advancing to term 7
I20260807 01:48:24.705600 32428 raft_consensus.cc:493] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Starting leader election (detected failure of leader 57bb4985157447f0a3ab680fa0026f12)
I20260807 01:48:24.756958 32549 raft_consensus.cc:3060] T 40f160d4dff84d8d9fb21b3489d01ff7 P fcf4e45defb94276bcfa68392711dade [term 7 FOLLOWER]: Advancing to term 8
W20260807 01:48:24.784685 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:24.775008 31908 raft_consensus.cc:697] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [term 7 LEADER]: Becoming Leader. State: Replica: 3478729505b046c2a0051c2d0130873f, State: Running, Role: LEADER
W20260807 01:48:24.793244 27172 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:24.790750 27102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37478d818c864abc83662e98299718d1" candidate_uuid: "3478729505b046c2a0051c2d0130873f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3262a9d183914b699227fdba0ca5bed3" is_pre_election: true
I20260807 01:48:24.792861 27105 raft_consensus.cc:1275] T 8dd88b3a4c37491bbe304dae1bdb08fb P 3262a9d183914b699227fdba0ca5bed3 [term 8 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260807 01:48:24.793741 32497 raft_consensus.cc:493] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 01:48:24.779932 32511 raft_consensus.cc:3060] T 21e1a5771ca04812aac6f5abed966ccd P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Advancing to term 10
I20260807 01:48:24.785486 32428 raft_consensus.cc:3060] T 31708844dc4f41feb6a7a2820cfc0e0b P fcf4e45defb94276bcfa68392711dade [term 5 FOLLOWER]: Advancing to term 6
I20260807 01:48:24.828068 27100 raft_consensus.cc:1275] T 879666f362ba4a2db3fdfe6738b37dec P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Refusing update from remote peer 3478729505b046c2a0051c2d0130873f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260807 01:48:24.825601 27102 raft_consensus.cc:2468] T 37478d818c864abc83662e98299718d1 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3478729505b046c2a0051c2d0130873f in term 7.
W20260807 01:48:24.829943 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 168071us
I20260807 01:48:24.754544 32043 raft_consensus.cc:515] T 3b3fc468354b48cf80eee526def18e6c P 3262a9d183914b699227fdba0ca5bed3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:24.841683 27287 leader_election.cc:400] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Vote denied by peer 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499) with higher term. Message: Invalid argument: T d8e18e65b8d2418cb159040f0fae8be3 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcf4e45defb94276bcfa68392711dade for earlier term 8. Current term is 9.
I20260807 01:48:24.843093 27287 leader_election.cc:403] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
W20260807 01:48:24.856418 27286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:24.780239 32529 consensus_queue.cc:260] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:24.894171 27287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 54228.2us
I20260807 01:48:24.895769 27109 raft_consensus.cc:2468] T c8799906440c4ef19cfbadc18cf32ef0 P 3262a9d183914b699227fdba0ca5bed3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb4985157447f0a3ab680fa0026f12 in term 7.
W20260807 01:48:24.905843 32687 log.cc:927] Time spent T 60a03ab231464be89d6cce9e6988cfba P 3262a9d183914b699227fdba0ca5bed3: Append to log took a long time: real 0.089s user 0.000s sys 0.001s
I20260807 01:48:24.841647 32497 raft_consensus.cc:515] T ed1089e78f8246fcb8af73c74af99b19 P 3262a9d183914b699227fdba0ca5bed3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 } }
I20260807 01:48:24.896286 32673 raft_consensus.cc:3060] T d8e18e65b8d2418cb159040f0fae8be3 P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Advancing to term 9
W20260807 01:48:24.908396 27063 leader_election.cc:336] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 1.983s (SENT)
W20260807 01:48:24.911435 28655 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.4.3:42429, user_credentials={real_user=slave}} blocked reactor thread for 237976us
I20260807 01:48:24.913882 27328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31708844dc4f41feb6a7a2820cfc0e0b" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:24.914719 27063 leader_election.cc:304] T 31708844dc4f41feb6a7a2820cfc0e0b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3262a9d183914b699227fdba0ca5bed3; no voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade
W20260807 01:48:24.915295 28660 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.24.4.1:35499, user_credentials={real_user=slave}} blocked reactor thread for 242749us
W20260807 01:48:24.907687 32574 consensus_meta.cc:287] T b18526f467154c49b830de9b1b03066f P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 0.826s user 0.007s sys 0.005s
I20260807 01:48:24.914121 28655 leader_election.cc:304] T 1c4e145f8f52431fa41b1f596e2f1133 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12, fcf4e45defb94276bcfa68392711dade; no voters:
I20260807 01:48:24.916683 28660 leader_election.cc:304] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
I20260807 01:48:24.917740 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54eb75662df043488517f09bff1dfcff" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
W20260807 01:48:24.916778 27063 leader_election.cc:336] T f27eab393b1f4f21aad32707749effa3 P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429): Timed out: RequestConsensusVote RPC to 127.24.4.3:42429 timed out after 2.324s (SENT)
I20260807 01:48:24.925561 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fabde7ae28974f7380cb7641ab9fd6b1" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:24.902194 32251 consensus_queue.cc:1059] T 2cfefe1ff8f14fb68678ac128e129d32 P 57bb4985157447f0a3ab680fa0026f12 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fcf4e45defb94276bcfa68392711dade" member_type: VOTER last_known_addr { host: "127.24.4.3" port: 42429 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260807 01:48:24.926342 27336 raft_consensus.cc:2468] T fabde7ae28974f7380cb7641ab9fd6b1 P fcf4e45defb94276bcfa68392711dade [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3262a9d183914b699227fdba0ca5bed3 in term 9.
I20260807 01:48:24.927559 28660 leader_election.cc:304] T 09799289505a4c188931e0f3ed0adfb3 P 57bb4985157447f0a3ab680fa0026f12 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57bb4985157447f0a3ab680fa0026f12; no voters: 3262a9d183914b699227fdba0ca5bed3, 3478729505b046c2a0051c2d0130873f
W20260807 01:48:24.932576 27289 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:24.947036 28661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 01:48:24.933131 32545 consensus_meta.cc:287] T bf9ccfe7f7e64481b739c6efef03b95a P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 0.740s user 0.009s sys 0.000s
W20260807 01:48:24.934567 32689 log.cc:927] Time spent T 6b0f72aee1954e3ab89690c7cfb93677 P 3478729505b046c2a0051c2d0130873f: Append to log took a long time: real 0.061s user 0.001s sys 0.000s
I20260807 01:48:24.937273 32691 raft_consensus.cc:3060] T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12 [term 6 FOLLOWER]: Advancing to term 7
W20260807 01:48:24.937810 28662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:24.938182 32574 raft_consensus.cc:493] T 879666f362ba4a2db3fdfe6738b37dec P 57bb4985157447f0a3ab680fa0026f12 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 01:48:24.944598 32555 consensus_meta.cc:287] T 60a03ab231464be89d6cce9e6988cfba P 57bb4985157447f0a3ab680fa0026f12: Time spent flushing consensus metadata: real 0.593s user 0.010s sys 0.000s
I20260807 01:48:24.926697 32318 leader_election.cc:290] T 965af9378c6a4617b1d50f229ffc2c5b P 3262a9d183914b699227fdba0ca5bed3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fcf4e45defb94276bcfa68392711dade (127.24.4.3:42429), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
W20260807 01:48:24.934244 32690 log.cc:927] Time spent T f27eab393b1f4f21aad32707749effa3 P 57bb4985157447f0a3ab680fa0026f12: Append to log took a long time: real 0.057s user 0.001s sys 0.000s
I20260807 01:48:24.916466 31908 consensus_queue.cc:237] T 94962ef95ca14973bc39a17ed76c0087 P 3478729505b046c2a0051c2d0130873f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3262a9d183914b699227fdba0ca5bed3" member_type: VOTER last_known_addr { host: "127.24.4.1" port: 35499 } } peers { permanent_uuid: "3478729505b046c2a0051c2d0130873f" member_type: VOTER last_known_addr { host: "127.24.4.2" port: 34855 } } peers { permanent_uuid: "57bb4985157447f0a3ab680fa0026f12" member_type: VOTER last_known_addr { host: "127.24.4.4" port: 46225 } attrs { promote: false } }
W20260807 01:48:24.955003 28661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 01:48:24.945924 32524 raft_consensus.cc:2749] T 94962ef95ca14973bc39a17ed76c0087 P 57bb4985157447f0a3ab680fa0026f12 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260807 01:48:24.965245 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd88b3a4c37491bbe304dae1bdb08fb" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:24.970850 27325 raft_consensus.cc:2393] T 8dd88b3a4c37491bbe304dae1bdb08fb P fcf4e45defb94276bcfa68392711dade [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3262a9d183914b699227fdba0ca5bed3 in current term 8: Already voted for candidate 3478729505b046c2a0051c2d0130873f in this term.
I20260807 01:48:24.971663 28706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00395257beb442e892a4ec18c4c30d95" candidate_uuid: "fcf4e45defb94276bcfa68392711dade" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "57bb4985157447f0a3ab680fa0026f12"
I20260807 01:48:24.945629 32541 leader_election.cc:290] T 00395257beb442e892a4ec18c4c30d95 P fcf4e45defb94276bcfa68392711dade [CANDIDATE]: Term 9 election: Requested vote from peers 3262a9d183914b699227fdba0ca5bed3 (127.24.4.1:35499), 3478729505b046c2a0051c2d0130873f (127.24.4.2:34855), 57bb4985157447f0a3ab680fa0026f12 (127.24.4.4:46225)
I20260807 01:48:24.961920 27336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53d8de481420436498aadc9930032da5" candidate_uuid: "3262a9d183914b699227fdba0ca5bed3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "fcf4e45defb94276bcfa68392711dade" is_pre_election: true
I20260807 01:48:25.010584 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "